09/01/2025 01:34:41.128 7fbb41d10440 [contrail-api] [INFO]: SANDESH: Logging: SYSLOG: [None] -> [LOG_LOCAL0] 09/01/2025 01:34:41.128 7fbb41d10440 [contrail-api] [INFO]: SANDESH: Trace: PRINT: [None] -> [False] 09/01/2025 01:34:41.128 7fbb41d10440 [contrail-api] [INFO]: SANDESH: Flow Logging: [None] -> [False] 09/01/2025 01:34:41.128 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Connecting to zookeeper on 10.0.0.50:2181 09/01/2025 01:34:41.128 7fbb41d10440 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:34:41.129 7fbb3d478860 [contrail-api] [ERROR]: Cannot write http_port 8084 to /tmp/contrail-api.0.http_port 09/01/2025 01:34:41.129 7fbb3d478860 [contrail-api] [ERROR]: Starting Introspect on HTTP Port 8084 09/01/2025 01:34:41.129 7fbb3d478c20 [contrail-api] [INFO]: Processing event[EvStart] in state[none] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [DEBUG]: Add uve in the [ObjectGeneratorInfo:SandeshModuleClientTrace] map 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Sandesh Client: Event[EvStart] => State[none] -> State[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[] connection down >>, ] >> 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.130 7fbb3d478c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 2 max_sm_queue_count = 2 >> 09/01/2025 01:34:41.131 7fbb3d478c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.269 7fbb3d478c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.270 7fbb3d478c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 09/01/2025 01:34:41.270 7fbb3d478c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.471 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Connecting to database on ['10.0.0.50:9041'] 09/01/2025 01:34:41.471 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: VNCCassandra started with driver cql 09/01/2025 01:34:41.472 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: initializing cassandra cluster 09/01/2025 01:34:41.472 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster before create 09/01/2025 01:34:41.472 7fbb3d478cc0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.473 7fbb3d478cc0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:34:41.473 7fbb3d478cc0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.480 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster created 09/01/2025 01:34:41.548 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster DB connected 09/01/2025 01:34:41.549 7fbb3e027e00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.550 7fbb3e027e00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Zookeeper:Zookeeper[], Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:34:41.550 7fbb3e027e00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.559 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster init KS 09/01/2025 01:34:41.559 7fbb3e027d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.559 7fbb3e027d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:34:41.559 7fbb3e027d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:41.584 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:41.754 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'config_db_uuid', created 09/01/2025 01:34:41.759 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:42.116 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:42.279 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'config_db_uuid' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/01/2025 01:34:42.285 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:43.157 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:43.295 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'useragent', created 09/01/2025 01:34:43.298 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:43.759 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:43.814 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: keyspace 'useragent' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 09/01/2025 01:34:43.818 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:43.818 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster KS are ready 09/01/2025 01:34:43.847 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: CassandraDriverCQL.__init__Cluster CF are ready 09/01/2025 01:34:44.200 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:44.352 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_shared_table', created 09/01/2025 01:34:44.356 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:44.949 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:45.040 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:34:45.052 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:45.176 7fbb3d1dfc20 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:34:45.177 7fbb3d1dfc20 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:34:45.178 7fbb3d1dfc20 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:34:45.178 7fbb3d1dfc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:45.178 7fbb3d1dfc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:34:45.178 7fbb3d1dfc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:45.178 7fbb3d1dfc20 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:Cassandra[] connection down >>, ] >> 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 3 >> 09/01/2025 01:34:45.179 7fbb3d1dfc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:45.525 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:45.649 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'useragent_keyval_table', created 09/01/2025 01:34:45.653 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:45.895 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock acquired. Performing schema change... 09/01/2025 01:34:45.973 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: table 'useragent_keyval_table' fixed with properties {'gc_grace_seconds': 864000} 09/01/2025 01:34:45.977 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Zookeeper lock released. 09/01/2025 01:34:45.977 7fbb3e027040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:45.977 7fbb3e027040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:45.977 7fbb3e027040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.055 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//'] 09/01/2025 01:34:46.063 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ drainer connection down 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:34:46.064 7fbb3d1df900 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.077 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ drainer connection ESTABLISHED 09/01/2025 01:34:46.077 7fbb3d1df900 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.077 7fbb3d1df900 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:46.077 7fbb3d1df900 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.089 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection down 09/01/2025 01:34:46.090 7fbb3decd5e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.090 7fbb3decd5e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:34:46.090 7fbb3decd5e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.096 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:34:46.101 7fbb3decd5e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.101 7fbb3decd5e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:46.102 7fbb3decd5e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.102 7fbb3decd900 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection down 09/01/2025 01:34:46.104 7fbb3decdb80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.104 7fbb3decdb80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail], Database:RabbitMQ[] connection down >>, ] >> 09/01/2025 01:34:46.104 7fbb3decdb80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.107 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config']: Unknown id: global_system_config default-global-system-config 09/01/2025 01:34:46.114 7fbb3decd900 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: RabbitMQ publish connection ESTABLISHED 09/01/2025 01:34:46.114 7fbb3decdea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.114 7fbb3decdea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:46.114 7fbb3decdea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:46.145 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-global-vrouter-config']: Unknown id: global_vrouter_config default-global-system-config:default-global-vrouter-config 09/01/2025 01:34:46.242 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain']: Unknown id: domain default-domain 09/01/2025 01:34:46.522 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-fabric']: Unknown id: fabric default-global-system-config:default-fabric 09/01/2025 01:34:46.577 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-policy-management']: Unknown id: policy_management default-policy-management 09/01/2025 01:34:46.609 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-policy-management', 'default-application-policy-set']: Unknown id: application_policy_set default-policy-management:default-application-policy-set 09/01/2025 01:34:46.664 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric']: Unknown id: virtual_network default-domain:default-project:ip-fabric 09/01/2025 01:34:46.746 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric']: Unknown id: routing_instance default-domain:default-project:ip-fabric:ip-fabric 09/01/2025 01:34:46.781 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'ip-fabric', '__default__']: Unknown id: routing_instance default-domain:default-project:ip-fabric:__default__ 09/01/2025 01:34:46.834 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__link_local__']: Unknown id: virtual_network default-domain:default-project:__link_local__ 09/01/2025 01:34:46.918 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__link_local__', '__link_local__']: Unknown id: routing_instance default-domain:default-project:__link_local__:__link_local__ 09/01/2025 01:34:46.955 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network']: Unknown id: virtual_network default-domain:default-project:dci-network 09/01/2025 01:34:47.062 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network', 'dci-network']: Unknown id: routing_instance default-domain:default-project:dci-network:dci-network 09/01/2025 01:34:47.154 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'dci-network', '__default__']: Unknown id: routing_instance default-domain:default-project:dci-network:__default__ 09/01/2025 01:34:47.208 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'system-defined-bottle-request-size-limit']: Unknown id: alarm default-global-system-config:system-defined-bottle-request-size-limit 09/01/2025 01:34:47.245 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network']: Unknown id: routing_instance default-domain:default-project:default-virtual-network:default-virtual-network 09/01/2025 01:34:47.276 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', '__no_rule__']: Unknown id: security_group default-domain:default-project:__no_rule__ 09/01/2025 01:34:47.303 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-discovery-service-assignment']: Unknown id: discovery_service_assignment default-discovery-service-assignment 09/01/2025 01:34:47.340 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-global-system-config', 'default-global-qos-config']: Unknown id: global_qos_config default-global-system-config:default-global-qos-config 09/01/2025 01:34:47.363 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'service-chain-flat-ipam']: Unknown id: network_ipam default-domain:default-project:service-chain-flat-ipam 09/01/2025 01:34:47.437 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['label']: Unknown id: tag_type label 09/01/2025 01:34:47.486 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['application']: Unknown id: tag_type application 09/01/2025 01:34:47.511 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['tier']: Unknown id: tag_type tier 09/01/2025 01:34:47.525 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['deployment']: Unknown id: tag_type deployment 09/01/2025 01:34:47.543 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['site']: Unknown id: tag_type site 09/01/2025 01:34:47.557 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['neutron_fwaas']: Unknown id: tag_type neutron_fwaas 09/01/2025 01:34:47.574 7fbb41d10440 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['neutron_tag']: Unknown id: tag_type neutron_tag 09/01/2025 01:34:47.585 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Executing db_resync on api-server instance 10.0.0.50_0 09/01/2025 01:34:47.585 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Contrail Version is(nightly) 09/01/2025 01:34:47.601 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: zk_sync_node data (None) 09/01/2025 01:34:47.602 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DBERESYNC: running dbe-resync 09/01/2025 01:34:47.613 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type tag_type len 7 09/01/2025 01:34:47.613 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for tag_type 09/01/2025 01:34:47.714 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Starting consumer greenlet 09/01/2025 01:34:47.715 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:34:47.715 7fbb3d0e3540 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Starting publisher greenlet 09/01/2025 01:34:47.715 7fbb3d0e3540 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Publisher connection 09/01/2025 01:34:47.716 7fbb41d10440 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Added consumer: amqp_request.cn-jenkins-deploy-platform-juju-k8s-1454-1.dummy 09/01/2025 01:34:47.716 7fbb41d10440 [contrail-api] [DEBUG]: Add uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:34:47.717 7fbb3d0e3ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.717 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ConfigApiWorkerTrace: data = << name = worker-0 worker_id = 0 introspect_port = 8084 admin_port = 8095 introspect_url = http://10.0.0.50:8084 >> 09/01/2025 01:34:47.717 7fbb3d0e3ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.718 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for tag_type 09/01/2025 01:34:47.718 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type policy_management len 1 09/01/2025 01:34:47.718 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for policy_management 09/01/2025 01:34:47.721 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for policy_management 09/01/2025 01:34:47.721 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type domain len 1 09/01/2025 01:34:47.721 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for domain 09/01/2025 01:34:47.721 7fbb3d0e3540 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Publisher connection established 09/01/2025 01:34:47.722 7fbb3d0e3400 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Consumer connection established 09/01/2025 01:34:47.726 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Created consumers ['amqp_request.cn-jenkins-deploy-platform-juju-k8s-1454-1.dummy'] 09/01/2025 01:34:47.727 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for domain 09/01/2025 01:34:47.727 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type routing_instance len 6 09/01/2025 01:34:47.727 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for routing_instance 09/01/2025 01:34:47.733 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Ensuring Consumer connection 09/01/2025 01:34:47.736 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for routing_instance 09/01/2025 01:34:47.736 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type virtual_network len 4 09/01/2025 01:34:47.736 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for virtual_network 09/01/2025 01:34:47.738 7fbb3d0e3400 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: KombuAmqpClient: Consumer connection established 09/01/2025 01:34:47.744 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: Created consumers ['amqp_request.cn-jenkins-deploy-platform-juju-k8s-1454-1.dummy'] 09/01/2025 01:34:47.752 7fbb3ceb7360 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:47.753 7fbb3ceb74a0 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:47.753 7fbb3ceb7e00 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:47.753 7fbb3ceb7cc0 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:47.753 7fbb3ceb7b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:default-virtual-network elements = { fq_name : ["default-domain", "default-project", "default-virtual-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17465821102750450049, "uuid_lslong": 9715519765744272016}, "enable": true, "created": "2025-09-01T01:34:46.451542", "last_modified": "2025-09-01T01:34:47.741349", "description": null, "user_visible": true, "creator": null}is_shared : falseparent_type : "project"parent_uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "f2630fc1-4bf6-4181-86d4-75c07d794290"virtual_network_network_id : 1 } deleted = False >> 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:__link_local__ elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "__link_local__"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10794140686179846417, "uuid_lslong": 13284926956428896337}, "enable": true, "created": "2025-09-01T01:34:46.855771", "last_modified": "2025-09-01T01:34:47.743186", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : falseis_shared : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"parent_uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "95cc7dc3-a757-4911-b85d-8aae28f4a851"virtual_network_category : "tenant"virtual_network_network_id : 3 } deleted = False >> 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:ip-fabric elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "ip-fabric"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16176487876338597955, "uuid_lslong": 12358777572459629502}, "enable": true, "created": "2025-09-01T01:34:46.683844", "last_modified": "2025-09-01T01:34:47.743615", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : trueis_shared : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"parent_uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "e07e6e04-4e93-4043-ab83-32b8f8854bbe"virtual_network_category : "tenant"virtual_network_network_id : 2 } deleted = False >> 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:dci-network elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "dci-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 14237869853516382962, "uuid_lslong": 12482820789020571781}, "enable": true, "created": "2025-09-01T01:34:46.999610", "last_modified": "2025-09-01T01:34:47.745089", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : falseis_shared : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"parent_uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "c5971332-4116-42f2-ad3b-e35eec6eb885"virtual_network_category : "tenant"virtual_network_network_id : 4 } deleted = False >> 09/01/2025 01:34:47.754 7fbb3ceb7b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.754 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for virtual_network 09/01/2025 01:34:47.755 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type fabric len 1 09/01/2025 01:34:47.755 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for fabric 09/01/2025 01:34:47.757 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for fabric 09/01/2025 01:34:47.757 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_vrouter_config len 1 09/01/2025 01:34:47.757 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_vrouter_config 09/01/2025 01:34:47.758 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_vrouter_config 09/01/2025 01:34:47.758 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_qos_config len 1 09/01/2025 01:34:47.759 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_qos_config 09/01/2025 01:34:47.760 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_qos_config 09/01/2025 01:34:47.760 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type global_system_config len 1 09/01/2025 01:34:47.760 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for global_system_config 09/01/2025 01:34:47.761 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for global_system_config 09/01/2025 01:34:47.761 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type security_group len 1 09/01/2025 01:34:47.761 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for security_group 09/01/2025 01:34:47.763 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for security_group 09/01/2025 01:34:47.763 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type project len 1 09/01/2025 01:34:47.763 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for project 09/01/2025 01:34:47.765 7fbb3d0e3c20 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Entry not found in Cassandra or ZooKeeper for ['default-domain', 'default-project', 'default-application-policy-set']: Unknown id: application_policy_set default-domain:default-project:default-application-policy-set 09/01/2025 01:34:47.776 7fbb3ceb7680 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:34:47.776 7fbb3ceb7b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.776 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project elements = { fq_name : ["default-domain", "default-project"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6187309951710415863, "uuid_lslong": 10932287968772668836}, "enable": true, "created": "2025-09-01T01:34:46.292044", "last_modified": "2025-09-01T01:34:46.292044", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4" } deleted = False >> 09/01/2025 01:34:47.776 7fbb3ceb7b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.776 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for project 09/01/2025 01:34:47.776 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type network_ipam len 2 09/01/2025 01:34:47.776 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for network_ipam 09/01/2025 01:34:47.784 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for network_ipam 09/01/2025 01:34:47.784 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type namespace len 1 09/01/2025 01:34:47.784 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for namespace 09/01/2025 01:34:47.786 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for namespace 09/01/2025 01:34:47.786 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type application_policy_set len 1 09/01/2025 01:34:47.786 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for application_policy_set 09/01/2025 01:34:47.789 7fbb3ceb72c0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:34:47.790 7fbb3ceb79a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.790 7fbb3ceb79a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:default-application-policy-set elements = { all_applications : truefq_name : ["default-policy-management", "default-application-policy-set"]global_vrouter_config_refs : [{"to": ["default-global-system-config", "default-global-vrouter-config"], "attr": null, "uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0"}]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16473062432589499027, "uuid_lslong": 13656848237090517172}, "enable": true, "created": "2025-09-01T01:34:46.635352", "last_modified": "2025-09-01T01:34:46.646597", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e49c1301-2154-4a93-bd86-df1d533330b4" } deleted = False >> 09/01/2025 01:34:47.790 7fbb3ceb79a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.790 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for application_policy_set 09/01/2025 01:34:47.790 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type discovery_service_assignment len 1 09/01/2025 01:34:47.790 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for discovery_service_assignment 09/01/2025 01:34:47.792 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for discovery_service_assignment 09/01/2025 01:34:47.792 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DB walk: obj_type alarm len 1 09/01/2025 01:34:47.792 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Start DB Resync for alarm 09/01/2025 01:34:47.795 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Finished DB Resync for alarm 09/01/2025 01:34:47.798 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: DBERESYNC: Completed successfully 09/01/2025 01:34:47.805 7fbb3d1f6180 [contrail-api] [INFO]: __default__ [SYS_INFO]: VncApiInfo: Cassandra DB walk completed. 09/01/2025 01:34:47.816 7fbb3d1f6180 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Time elapsed in resyncing db: 0:00:00.230508 09/01/2025 01:34:47.817 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'obj_dict': {'autonomous_system': 64512, 'config_version': '1.0', 'enable_4byte_as': False, 'enable_security_policy_draft': False, 'fq_name': ['default-global-system-config'], 'ibgp_auto_mesh': True, 'id_perms': {'created': '2025-09-01T01:34:46.126813', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.126813', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11642697718081793789, 'uuid_mslong': 8136497840866148553}}, 'igmp_enable': False, 'mac_aging_time': 300, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '70eaa5ad-608f-40c9-a193-2be391842efd'}, 'oper': 'CREATE', 'request-id': 'req-af6c7f4d-3ad8-46de-9d14-d24deec1cc20', 'type': 'global_system_config', 'uuid': '70eaa5ad-608f-40c9-a193-2be391842efd'} 09/01/2025 01:34:47.822 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'obj_dict': {'enable_security_logging': True, 'encryption_mode': 'none', 'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'global_max_vmi_flows': 0, 'id_perms': {'created': '2025-09-01T01:34:46.212133', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.212133', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12288031742954674112, 'uuid_mslong': 9774649233780066708}}, 'max_vm_flows_percent': 0, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'}, 'oper': 'CREATE', 'request-id': 'req-87c92b1f-d14a-4a19-8d03-43781db1169e', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:34:47.823 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain'], 'obj_dict': {'fq_name': ['default-domain'], 'id_perms': {'created': '2025-09-01T01:34:46.271808', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.271808', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9902634091162837009, 'uuid_mslong': 14845388617369141669}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011'}, 'oper': 'CREATE', 'request-id': 'req-dc450a80-4582-41b0-95d5-3870f0096e81', 'type': 'domain', 'uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011'} 09/01/2025 01:34:47.823 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project'], 'obj_dict': {'_enable_security_policy_draft': False, '_type': 'project', '_uuid': None, 'fq_name': ['default-domain', 'default-project'], 'id_perms': {'created': '2025-09-01T01:34:46.292044', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.292044', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10932287968772668836, 'uuid_mslong': 6187309951710415863}}, 'name': 'default-project', 'parent_type': 'domain', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4'}, 'oper': 'CREATE', 'request-id': 'req-8cd35147-b875-4b41-a5f4-09b421e5717c', 'type': 'project', 'uuid': '55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4'} 09/01/2025 01:34:47.824 7fbb3d1f6180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.824 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project elements = { _enable_security_policy_draft : false_type : "project"_uuid : nullfq_name : ["default-domain", "default-project"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6187309951710415863, "uuid_lslong": 10932287968772668836}, "enable": true, "created": "2025-09-01T01:34:46.292044", "last_modified": "2025-09-01T01:34:46.292044", "description": null, "user_visible": true, "creator": null}name : "default-project"parent_type : "domain"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4" } deleted = False >> 09/01/2025 01:34:47.824 7fbb3d1f6180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.824 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'obj_dict': {'_type': 'network-ipam', '_uuid': None, 'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'id_perms': {'created': '2025-09-01T01:34:46.394226', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.394226', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10581292310553591075, 'uuid_mslong': 9875949571202367850}}, 'name': 'default-network-ipam', 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '890e6bcf-5b30-416a-92d8-4d4b5b1e7d23'}, 'oper': 'CREATE', 'request-id': 'req-29226334-3007-4833-80d0-a2b4d85d690c', 'type': 'network_ipam', 'uuid': '890e6bcf-5b30-416a-92d8-4d4b5b1e7d23'} 09/01/2025 01:34:47.825 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network'], 'obj_dict': {'_fabric_snat': False, '_flood_unknown_unicast': False, '_igmp_enable': False, '_is_provider_network': False, '_layer2_control_word': False, '_mac_aging_time': 300, '_mac_ip_learning_enable': False, '_mac_learning_enabled': False, '_pbb_etree_enable': False, '_pbb_evpn_enable': False, '_port_security_enabled': True, '_type': 'virtual-network', '_uuid': None, '_virtual_network_category': 'tenant', 'fq_name': ['default-domain', 'default-project', 'default-virtual-network'], 'id_perms': {'created': '2025-09-01T01:34:46.451542', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.451542', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9715519765744272016, 'uuid_mslong': 17465821102750450049}}, 'name': 'default-virtual-network', 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f2630fc1-4bf6-4181-86d4-75c07d794290', 'virtual_network_network_id': 1}, 'oper': 'CREATE', 'request-id': 'req-96d3e0db-61ab-443f-9d75-d6816a8ca8c3', 'type': 'virtual_network', 'uuid': 'f2630fc1-4bf6-4181-86d4-75c07d794290'} 09/01/2025 01:34:47.826 7fbb3ceb79a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.826 7fbb3ceb79a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:default-virtual-network elements = { _fabric_snat : false_flood_unknown_unicast : false_igmp_enable : false_is_provider_network : false_layer2_control_word : false_mac_aging_time : 300_mac_ip_learning_enable : false_mac_learning_enabled : false_pbb_etree_enable : false_pbb_evpn_enable : false_port_security_enabled : true_type : "virtual-network"_uuid : null_virtual_network_category : "tenant"fq_name : ["default-domain", "default-project", "default-virtual-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17465821102750450049, "uuid_lslong": 9715519765744272016}, "enable": true, "created": "2025-09-01T01:34:46.451542", "last_modified": "2025-09-01T01:34:46.451542", "description": null, "user_visible": true, "creator": null}name : "default-virtual-network"parent_type : "project"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "f2630fc1-4bf6-4181-86d4-75c07d794290"virtual_network_network_id : 1 } deleted = False >> 09/01/2025 01:34:47.826 7fbb3ceb79a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.828 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-namespace'], 'obj_dict': {'_type': 'namespace', '_uuid': None, 'fq_name': ['default-domain', 'default-namespace'], 'id_perms': {'created': '2025-09-01T01:34:46.513685', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.513685', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10565767707462785886, 'uuid_mslong': 9151343710783947602}}, 'name': 'default-namespace', 'parent_type': 'domain', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7f001a9e-7ac8-4352-92a1-25c0069cdb5e'}, 'oper': 'CREATE', 'request-id': 'req-0c3d9639-052d-45e6-ba28-f6e96454b73c', 'type': 'namespace', 'uuid': '7f001a9e-7ac8-4352-92a1-25c0069cdb5e'} 09/01/2025 01:34:47.829 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-fabric'], 'obj_dict': {'disable_vlan_vn_uniqueness_check': False, 'fabric_enterprise_style': True, 'fq_name': ['default-global-system-config', 'default-fabric'], 'id_perms': {'created': '2025-09-01T01:34:46.563867', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.563867', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13276833899797486968, 'uuid_mslong': 13359285646941373286}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b965b783-eeea-4366-b840-ca16934b1d78'}, 'oper': 'CREATE', 'request-id': 'req-909a80cc-0008-4f6d-902a-ac5afd4b755d', 'type': 'fabric', 'uuid': 'b965b783-eeea-4366-b840-ca16934b1d78'} 09/01/2025 01:34:47.830 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management'], 'obj_dict': {'fq_name': ['default-policy-management'], 'id_perms': {'created': '2025-09-01T01:34:46.603878', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.603878', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10980116163776804647, 'uuid_mslong': 5354730006820045128}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4a4fd292-14a2-4148-9861-3579ce70f727'}, 'oper': 'CREATE', 'request-id': 'req-5207c32c-359f-4972-b85b-dccd49d2117f', 'type': 'policy_management', 'uuid': '4a4fd292-14a2-4148-9861-3579ce70f727'} 09/01/2025 01:34:47.830 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'fq_name': ['default-policy-management', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:34:46.635352', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.635352', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13656848237090517172, 'uuid_mslong': 16473062432589499027}}, 'parent_type': 'policy-management', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e49c1301-2154-4a93-bd86-df1d533330b4'}, 'oper': 'CREATE', 'request-id': 'req-f24c290f-94b9-4223-abcb-f31a25421aa8', 'type': 'application_policy_set', 'uuid': 'e49c1301-2154-4a93-bd86-df1d533330b4'} 09/01/2025 01:34:47.831 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'default-application-policy-set'], 'oper': 'UPDATE', 'request-id': 'req-fad52f0f-524a-4aca-ab3a-b18283b216ee', 'type': 'application_policy_set', 'uuid': 'e49c1301-2154-4a93-bd86-df1d533330b4'} 09/01/2025 01:34:47.832 7fbb3decde00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.832 7fbb3decde00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:default-application-policy-set elements = { all_applications : truefq_name : ["default-policy-management", "default-application-policy-set"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16473062432589499027, "uuid_lslong": 13656848237090517172}, "enable": true, "created": "2025-09-01T01:34:46.635352", "last_modified": "2025-09-01T01:34:46.635352", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e49c1301-2154-4a93-bd86-df1d533330b4" } deleted = False >> 09/01/2025 01:34:47.832 7fbb3decde00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.834 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', 'ip-fabric'], 'id_perms': {'created': '2025-09-01T01:34:46.683844', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.683844', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12358777572459629502, 'uuid_mslong': 16176487876338597955}}, 'igmp_enable': False, 'is_provider_network': True, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': 'e07e6e04-4e93-4043-ab83-32b8f8854bbe', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 2}, 'oper': 'CREATE', 'request-id': 'req-342cd737-dd14-461f-9a89-7ba79e3d9fe1', 'type': 'virtual_network', 'uuid': 'e07e6e04-4e93-4043-ab83-32b8f8854bbe'} 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:default-application-policy-set elements = { all_applications : truefq_name : ["default-policy-management", "default-application-policy-set"]global_vrouter_config_refs : [{"to": ["default-global-system-config", "default-global-vrouter-config"], "attr": null, "uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0"}]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16473062432589499027, "uuid_lslong": 13656848237090517172}, "enable": true, "created": "2025-09-01T01:34:46.635352", "last_modified": "2025-09-01T01:34:46.646597", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e49c1301-2154-4a93-bd86-df1d533330b4" } deleted = False >> 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:ip-fabric elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "ip-fabric"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16176487876338597955, "uuid_lslong": 12358777572459629502}, "enable": true, "created": "2025-09-01T01:34:46.683844", "last_modified": "2025-09-01T01:34:46.683844", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : truelayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "e07e6e04-4e93-4043-ab83-32b8f8854bbe"virtual_network_category : "tenant"virtual_network_network_id : 2 } deleted = False >> 09/01/2025 01:34:47.835 7fbb3decde00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.837 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-09-01T01:34:46.771456', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.771456', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13112810308037457455, 'uuid_mslong': 2523496361995553647}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'}, 'oper': 'CREATE', 'request-id': 'req-ee749314-cd34-46b3-b8c4-a4feaf79282f', 'type': 'routing_instance', 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'} 09/01/2025 01:34:47.838 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__'], 'id_perms': {'created': '2025-09-01T01:34:46.817772', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.817772', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10579553307854272516, 'uuid_mslong': 3636760202386622067}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': False, 'uuid': '32785e22-bc88-4673-92d2-1fae3cf67804'}, 'oper': 'CREATE', 'request-id': 'req-5dd43704-9a58-461e-8cd0-866c534ab614', 'type': 'routing_instance', 'uuid': '32785e22-bc88-4673-92d2-1fae3cf67804'} 09/01/2025 01:34:47.838 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__link_local__'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', '__link_local__'], 'id_perms': {'created': '2025-09-01T01:34:46.855771', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.855771', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13284926956428896337, 'uuid_mslong': 10794140686179846417}}, 'igmp_enable': False, 'is_provider_network': False, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': '95cc7dc3-a757-4911-b85d-8aae28f4a851', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 3}, 'oper': 'CREATE', 'request-id': 'req-674e330f-b42c-4e83-af98-52879da1c58c', 'type': 'virtual_network', 'uuid': '95cc7dc3-a757-4911-b85d-8aae28f4a851'} 09/01/2025 01:34:47.839 7fbb3decde00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.839 7fbb3decde00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:__link_local__ elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "__link_local__"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10794140686179846417, "uuid_lslong": 13284926956428896337}, "enable": true, "created": "2025-09-01T01:34:46.855771", "last_modified": "2025-09-01T01:34:46.855771", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "95cc7dc3-a757-4911-b85d-8aae28f4a851"virtual_network_category : "tenant"virtual_network_network_id : 3 } deleted = False >> 09/01/2025 01:34:47.839 7fbb3decde00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.841 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__link_local__', '__link_local__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', '__link_local__', '__link_local__'], 'id_perms': {'created': '2025-09-01T01:34:46.935974', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.935974', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10617527933964632087, 'uuid_mslong': 7550229057685177839}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '68c7cd53-3c28-45ef-9359-0966e0bfc017'}, 'oper': 'CREATE', 'request-id': 'req-9584bd04-c392-4a64-86b8-36fa0d892932', 'type': 'routing_instance', 'uuid': '68c7cd53-3c28-45ef-9359-0966e0bfc017'} 09/01/2025 01:34:47.841 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network'], 'obj_dict': {'fabric_snat': False, 'flood_unknown_unicast': False, 'fq_name': ['default-domain', 'default-project', 'dci-network'], 'id_perms': {'created': '2025-09-01T01:34:46.999610', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:46.999610', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12482820789020571781, 'uuid_mslong': 14237869853516382962}}, 'igmp_enable': False, 'is_provider_network': False, 'layer2_control_word': False, 'mac_aging_time': 300, 'mac_ip_learning_enable': False, 'mac_learning_enabled': False, 'parent_type': 'project', 'pbb_etree_enable': False, 'pbb_evpn_enable': False, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'uuid': 'c5971332-4116-42f2-ad3b-e35eec6eb885', 'virtual_network_category': 'tenant', 'virtual_network_network_id': 4}, 'oper': 'CREATE', 'request-id': 'req-2b5389e4-42ff-4e7e-a449-9e71b8dd94d8', 'type': 'virtual_network', 'uuid': 'c5971332-4116-42f2-ad3b-e35eec6eb885'} 09/01/2025 01:34:47.842 7fbb3decde00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.842 7fbb3decde00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:dci-network elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "dci-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 14237869853516382962, "uuid_lslong": 12482820789020571781}, "enable": true, "created": "2025-09-01T01:34:46.999610", "last_modified": "2025-09-01T01:34:46.999610", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falseparent_type : "project"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "c5971332-4116-42f2-ad3b-e35eec6eb885"virtual_network_category : "tenant"virtual_network_network_id : 4 } deleted = False >> 09/01/2025 01:34:47.842 7fbb3decde00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.844 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'id_perms': {'created': '2025-09-01T01:34:47.119338', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.119338', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10294142737327693945, 'uuid_mslong': 15470973540864967106}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': 'd6b3f077-78b9-41c2-8edc-24476595a079'}, 'oper': 'CREATE', 'request-id': 'req-9cd18b62-a936-41c1-813b-bfe0b1b48da4', 'type': 'routing_instance', 'uuid': 'd6b3f077-78b9-41c2-8edc-24476595a079'} 09/01/2025 01:34:47.844 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'id_perms': {'created': '2025-09-01T01:34:47.180890', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.180890', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10526762196409238221, 'uuid_mslong': 11159808313272516628}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': False, 'uuid': '9adf9a88-a20f-4014-9216-9271c7fbc2cd'}, 'oper': 'CREATE', 'request-id': 'req-8ff0732c-5aad-4875-8dbf-4efdc6fe0302', 'type': 'routing_instance', 'uuid': '9adf9a88-a20f-4014-9216-9271c7fbc2cd'} 09/01/2025 01:34:47.845 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bottle-request-size-limit'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'UveConfigReq.err_info.*.', 'operand2': {'json_value': 'True'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'fq_name': ['default-global-system-config', 'system-defined-bottle-request-size-limit'], 'id_perms': {'created': '2025-09-01T01:34:47.223109', 'description': 'Bottle request size limit exceeded.', 'last_modified': '2025-09-01T01:34:47.223109', 'uuid': {'uuid_lslong': 13369006820992206267, 'uuid_mslong': 12091023637871478040}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a7cbf1f0-0a2e-4518-b988-40df2503c5bb', 'uve_keys': {'uve_key': ['config-node']}}, 'oper': 'CREATE', 'request-id': 'req-7a292a98-40a6-4dbd-aa45-39c05a8156d2', 'type': 'alarm', 'uuid': 'a7cbf1f0-0a2e-4518-b988-40df2503c5bb'} 09/01/2025 01:34:47.846 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'id_perms': {'created': '2025-09-01T01:34:47.267877', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.267877', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11139779855341822345, 'uuid_mslong': 18214209377562740624}}, 'parent_type': 'virtual-network', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': 'fcc5deed-0637-4790-9a98-72c1e1b3a189'}, 'oper': 'CREATE', 'request-id': 'req-53e38ef1-c44d-486e-9a40-b1ff9a30204f', 'type': 'routing_instance', 'uuid': 'fcc5deed-0637-4790-9a98-72c1e1b3a189'} 09/01/2025 01:34:47.847 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__'], 'obj_dict': {'configured_security_group_id': 0, 'fq_name': ['default-domain', 'default-project', '__no_rule__'], 'id_perms': {'created': '2025-09-01T01:34:47.295154', 'creator': None, 'description': 'Security group with no rules', 'enable': True, 'last_modified': '2025-09-01T01:34:47.295154', 'permissions': None, 'user_visible': True, 'uuid': {'uuid_lslong': 13540808602676504644, 'uuid_mslong': 7084951279173782157}}, 'parent_type': 'project', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': []}, 'security_group_id': 8000001, 'uuid': '6252cda6-961a-4a8d-bbea-9dae8e162844'}, 'oper': 'CREATE', 'request-id': 'req-851c4730-7cb6-4a82-b953-27428a58b5ba', 'type': 'security_group', 'uuid': '6252cda6-961a-4a8d-bbea-9dae8e162844'} 09/01/2025 01:34:47.848 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-discovery-service-assignment'], 'obj_dict': {'fq_name': ['default-discovery-service-assignment'], 'id_perms': {'created': '2025-09-01T01:34:47.325859', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.325859', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11386590342312030178, 'uuid_mslong': 13227569861984340767}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b791c4b4-f13d-4b1f-9e05-4b966c66efe2'}, 'oper': 'CREATE', 'request-id': 'req-4039e4a8-ab26-43bd-b1b5-5932e594ea91', 'type': 'discovery_service_assignment', 'uuid': 'b791c4b4-f13d-4b1f-9e05-4b966c66efe2'} 09/01/2025 01:34:47.849 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-qos-config'], 'obj_dict': {'fq_name': ['default-global-system-config', 'default-global-qos-config'], 'id_perms': {'created': '2025-09-01T01:34:47.353862', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.353862', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10051971262811056665, 'uuid_mslong': 5615064957833005984}}, 'parent_type': 'global-system-config', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '4decb7d4-4a61-4ba0-8b7f-c69b1c49aa19'}, 'oper': 'CREATE', 'request-id': 'req-2e075fe5-c389-481c-aa59-bd9b58e1700d', 'type': 'global_qos_config', 'uuid': '4decb7d4-4a61-4ba0-8b7f-c69b1c49aa19'} 09/01/2025 01:34:47.850 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'service-chain-flat-ipam'], 'obj_dict': {'fq_name': ['default-domain', 'default-project', 'service-chain-flat-ipam'], 'id_perms': {'created': '2025-09-01T01:34:47.429488', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.429488', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9592969401448867083, 'uuid_mslong': 9349228989015214265}}, 'ipam_subnet_method': 'flat-subnet', 'ipam_subnets': {'subnets': [{'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': None, 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': None, 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 8}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}, {'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': None, 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': None, 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '::ffff', 'ip_prefix_len': 104}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}]}, 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '81bf223b-2e09-48b9-8521-12d84a8d250b'}, 'oper': 'CREATE', 'request-id': 'req-54970d8a-41bd-44bc-9475-40e34f210cb5', 'type': 'network_ipam', 'uuid': '81bf223b-2e09-48b9-8521-12d84a8d250b'} 09/01/2025 01:34:47.853 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['label'], 'obj_dict': {'display_name': 'label', 'fq_name': ['label'], 'id_perms': {'created': '2025-09-01T01:34:47.469867', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.469867', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 9261865816858579910, 'uuid_mslong': 1730848115726502302}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0000', 'uuid': '18053530-da08-499e-8088-c1e3933053c6'}, 'oper': 'CREATE', 'request-id': 'req-6914c927-e5bc-472a-aefb-c6cd92e370e9', 'type': 'tag_type', 'uuid': '18053530-da08-499e-8088-c1e3933053c6'} 09/01/2025 01:34:47.854 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['application'], 'obj_dict': {'display_name': 'application', 'fq_name': ['application'], 'id_perms': {'created': '2025-09-01T01:34:47.497134', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.497134', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11231281168373267591, 'uuid_mslong': 15029083005061121331}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0001', 'uuid': 'd092075d-9887-4533-9bdd-86b77d6c3c87'}, 'oper': 'CREATE', 'request-id': 'req-28e47dfd-3fd7-4838-a0cd-830c2c09fe89', 'type': 'tag_type', 'uuid': 'd092075d-9887-4533-9bdd-86b77d6c3c87'} 09/01/2025 01:34:47.856 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['tier'], 'obj_dict': {'display_name': 'tier', 'fq_name': ['tier'], 'id_perms': {'created': '2025-09-01T01:34:47.517848', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.517848', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11153661457680864111, 'uuid_mslong': 10639181158427478717}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0002', 'uuid': '93a5f6e5-1119-4abd-9ac9-c4004a08936f'}, 'oper': 'CREATE', 'request-id': 'req-6c145333-7b42-460a-9e98-7977085dd8bb', 'type': 'tag_type', 'uuid': '93a5f6e5-1119-4abd-9ac9-c4004a08936f'} 09/01/2025 01:34:47.858 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['deployment'], 'obj_dict': {'display_name': 'deployment', 'fq_name': ['deployment'], 'id_perms': {'created': '2025-09-01T01:34:47.535616', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.535616', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 10890644181307696771, 'uuid_mslong': 1808775309783289649}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0003', 'uuid': '191a0f8f-9a21-4f31-9723-572e8a052683'}, 'oper': 'CREATE', 'request-id': 'req-096136ed-d6ad-45dc-8f45-5f8e47d50c72', 'type': 'tag_type', 'uuid': '191a0f8f-9a21-4f31-9723-572e8a052683'} 09/01/2025 01:34:47.861 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['site'], 'obj_dict': {'display_name': 'site', 'fq_name': ['site'], 'id_perms': {'created': '2025-09-01T01:34:47.549820', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.549820', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11145087469694852610, 'uuid_mslong': 15493257459784896144}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0004', 'uuid': 'd7031b92-a826-4e90-9aab-4e0125d19e02'}, 'oper': 'CREATE', 'request-id': 'req-12e24555-e3be-402a-873e-54ef97b4a0f6', 'type': 'tag_type', 'uuid': 'd7031b92-a826-4e90-9aab-4e0125d19e02'} 09/01/2025 01:34:47.862 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['neutron_fwaas'], 'obj_dict': {'display_name': 'neutron_fwaas', 'fq_name': ['neutron_fwaas'], 'id_perms': {'created': '2025-09-01T01:34:47.564105', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.564105', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11238983973485769596, 'uuid_mslong': 13409499024259827909}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0005', 'uuid': 'ba181c50-e4f4-48c5-9bf8-e4608fed737c'}, 'oper': 'CREATE', 'request-id': 'req-ad563f08-2d18-4318-bbc1-fb6a3cc9f0f9', 'type': 'tag_type', 'uuid': 'ba181c50-e4f4-48c5-9bf8-e4608fed737c'} 09/01/2025 01:34:47.864 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['neutron_tag'], 'obj_dict': {'display_name': 'neutron_tag', 'fq_name': ['neutron_tag'], 'id_perms': {'created': '2025-09-01T01:34:47.582774', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.582774', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 10581893830717965902, 'uuid_mslong': 10902521930368763527}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0006', 'uuid': '974d89ee-7f89-4687-92da-705faed7864e'}, 'oper': 'CREATE', 'request-id': 'req-bd3511f7-a57a-46d0-b2e0-8cd8dd8f5803', 'type': 'tag_type', 'uuid': '974d89ee-7f89-4687-92da-705faed7864e'} 09/01/2025 01:34:47.865 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-network-ipam'], 'oper': 'UPDATE', 'request-id': 'req-038abd6b-728e-4081-bbd4-52c09f20e730', 'type': 'network_ipam', 'uuid': '890e6bcf-5b30-416a-92d8-4d4b5b1e7d23'} 09/01/2025 01:34:47.870 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'fq_name': ['default-domain', 'default-project', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:34:47.772938', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.772938', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11372720241568830556, 'uuid_mslong': 3513765528940005485}}, 'parent_type': 'project', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '30c36721-c07b-486d-9dd4-04cdf09ad45c'}, 'oper': 'CREATE', 'request-id': 'req-41c1700e-3ebd-4a8e-bebc-c91f1733d8de', 'type': 'application_policy_set', 'uuid': '30c36721-c07b-486d-9dd4-04cdf09ad45c'} 09/01/2025 01:34:47.870 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:34:47.871 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project'], 'oper': 'UPDATE', 'request-id': 'req-55c1d685-ab08-4155-a0bc-7b9a3dd36a4e', 'type': 'project', 'uuid': '55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4'} 09/01/2025 01:34:47.871 7fbb3d1f6180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.871 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:default-application-policy-set elements = { all_applications : truefq_name : ["default-domain", "default-project", "default-application-policy-set"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 3513765528940005485, "uuid_lslong": 11372720241568830556}, "enable": true, "created": "2025-09-01T01:34:47.772938", "last_modified": "2025-09-01T01:34:47.772938", "description": null, "user_visible": true, "creator": null}parent_type : "project"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "30c36721-c07b-486d-9dd4-04cdf09ad45c" } deleted = False >> 09/01/2025 01:34:47.871 7fbb3d1f6180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.873 7fbb3d1f6180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.874 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project elements = { fq_name : ["default-domain", "default-project"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6187309951710415863, "uuid_lslong": 10932287968772668836}, "enable": true, "created": "2025-09-01T01:34:46.292044", "last_modified": "2025-09-01T01:34:47.813239", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}quota : {"defaults": -1}uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4" } deleted = False >> 09/01/2025 01:34:47.874 7fbb3d1f6180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.960 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:47.960 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:47.960 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:47.961 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:47.962 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:47.970 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: analytics_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:47.973 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/analytics-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_analytics_node.py remote_ip = 10.0.0.50 body = {"analytics-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "analytics_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:47.987 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'analytics_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:47.982849', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:47.982849', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9616190303525458299, 'uuid_mslong': 606717431842292638}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '086b7e4a-2bff-439e-8573-9221de30d17b'}, 'oper': 'CREATE', 'request-id': 'req-d3f7ddc9-e350-4c30-9b15-fb283c653ab7', 'type': 'analytics_node', 'uuid': '086b7e4a-2bff-439e-8573-9221de30d17b'} 09/01/2025 01:34:47.987 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectCollectorInfo:ContrailConfigTrace] map 09/01/2025 01:34:47.987 7fbb3ceb70e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:47.988 7fbb3ceb70e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { analytics_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 606717431842292638, "uuid_lslong": 9616190303525458299}, "enable": true, "created": "2025-09-01T01:34:47.982849", "last_modified": "2025-09-01T01:34:47.982849", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "086b7e4a-2bff-439e-8573-9221de30d17b" } deleted = False >> 09/01/2025 01:34:47.988 7fbb3ceb70e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:48.543 7fbb3d0e3d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:34:48.546 7fbb3d0e3d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:34:48.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:48.788 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:48.792 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:48.793 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:48.794 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:48.800 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: analytics_alarm_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:48.802 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_alarm_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/analytics-alarm-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_analytics_alarm_node.py remote_ip = 10.0.0.50 body = {"analytics-alarm-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "analytics_alarm_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:48.989 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'analytics_alarm_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:48.962365', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:48.962365', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9281175083743715570, 'uuid_mslong': 4570147916412765694}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3f6c6d50-f8c6-45fe-80cd-5b909cc020f2'}, 'oper': 'CREATE', 'request-id': 'req-240b4324-e7a6-486f-865d-ed38fb92efa4', 'type': 'analytics_alarm_node', 'uuid': '3f6c6d50-f8c6-45fe-80cd-5b909cc020f2'} 09/01/2025 01:34:48.989 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectAnalyticsAlarmInfo:ContrailConfigTrace] map 09/01/2025 01:34:48.990 7fbb3ceb7cc0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:48.990 7fbb3ceb7cc0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { analytics_alarm_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 4570147916412765694, "uuid_lslong": 9281175083743715570}, "enable": true, "created": "2025-09-01T01:34:48.962365", "last_modified": "2025-09-01T01:34:48.962365", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "3f6c6d50-f8c6-45fe-80cd-5b909cc020f2" } deleted = False >> 09/01/2025 01:34:48.990 7fbb3ceb7cc0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.178 7fbb3d0e3d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:49.179 7fbb3d0e3d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:34:49.180 7fbb3d0e3d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.368 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 09/01/2025 01:34:49.369 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 09/01/2025 01:34:49.369 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 09/01/2025 01:34:49.369 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 09/01/2025 01:34:49.370 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:49.370 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 09/01/2025 01:34:49.370 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 09/01/2025 01:34:49.371 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 09/01/2025 01:34:49.375 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: bgp_router default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:49.379 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = bgp_router identifier_name = default-domain:default-project:ip-fabric:__default__:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/bgp-routers operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.50 body = {"bgp-router": {"uuid": null, "fq_name": ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "routing-instance", "bgp_router_parameters": {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.50", "address": "10.0.0.50", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:49.401 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'bgp_router_parameters': {'address': '10.0.0.50', 'address_families': {'family': ['route-target', 'inet-vpn', 'e-vpn', 'erm-vpn', 'inet6-vpn']}, 'admin_down': False, 'auth_data': None, 'autonomous_system': 64512, 'cluster_id': None, 'gateway_address': None, 'hold_time': 0, 'identifier': '10.0.0.50', 'ipv6_gateway_address': None, 'local_autonomous_system': None, 'port': 179, 'router_type': 'control-node', 'source_port': None, 'vendor': 'contrail'}, 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', '__default__', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:49.394090', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:49.394090', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11653439740259801885, 'uuid_mslong': 16887815640464049052}}, 'parent_type': 'routing-instance', 'parent_uuid': '32785e22-bc88-4673-92d2-1fae3cf67804', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ea5d92d9-7a25-439c-a1b9-55b3819b1b1d'}, 'oper': 'CREATE', 'request-id': 'req-6ebbff28-38a6-45b2-86c3-7f359051fb5c', 'type': 'bgp_router', 'uuid': 'ea5d92d9-7a25-439c-a1b9-55b3819b1b1d'} 09/01/2025 01:34:49.403 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectBgpRouter:ContrailConfigTrace] map 09/01/2025 01:34:49.403 7fbb3ceb7d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.403 7fbb3ceb7d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { bgp_router_parameters : {"admin_down": false, "vendor": "contrail", "cluster_id": null, "autonomous_system": 64512, "identifier": "10.0.0.50", "address": "10.0.0.50", "port": 179, "source_port": null, "hold_time": 0, "address_families": {"family": ["route-target", "inet-vpn", "e-vpn", "erm-vpn", "inet6-vpn"]}, "auth_data": null, "local_autonomous_system": null, "router_type": "control-node", "gateway_address": null, "ipv6_gateway_address": null}display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16887815640464049052, "uuid_lslong": 11653439740259801885}, "enable": true, "created": "2025-09-01T01:34:49.394090", "last_modified": "2025-09-01T01:34:49.394090", "description": null, "user_visible": true, "creator": null}parent_type : "routing-instance"parent_uuid : "32785e22-bc88-4673-92d2-1fae3cf67804"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "ea5d92d9-7a25-439c-a1b9-55b3819b1b1d" } deleted = False >> 09/01/2025 01:34:49.403 7fbb3ceb7d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_refs in resource bgp-router: 'control_node_zone_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource bgp-router: 'sub_cluster_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource bgp-router: 'bgp_router_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource bgp-router: 'tag_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource bgp-router: 'global_system_config_back_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource bgp-router: 'physical_router_back_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource bgp-router: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource bgp-router: 'bgp_as_a_service_back_refs' 09/01/2025 01:34:49.413 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource bgp-router: 'bgp_router_back_refs' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:49.469 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:49.470 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource global-system-config: 'bgp_router_refs' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:49.471 7fbb3d0e3d60 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:49.475 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 70eaa5ad-608f-40c9-a193-2be391842efd object_type = global_system_config identifier_name = default-global-system-config url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.50 body = {"uuid": "70eaa5ad-608f-40c9-a193-2be391842efd", "ref-uuid": "ea5d92d9-7a25-439c-a1b9-55b3819b1b1d"} domain = default-domain >> 09/01/2025 01:34:49.533 7fbb3d0e3d60 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 70eaa5ad-608f-40c9-a193-2be391842efd object_type = global_system_config identifier_name = default-global-system-config url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.50 body = {"type": "global-system-config", "uuid": "70eaa5ad-608f-40c9-a193-2be391842efd", "ref-type": "bgp-router", "ref-uuid": "ea5d92d9-7a25-439c-a1b9-55b3819b1b1d", "ref-fq-name": ["default-domain", "default-project", "ip-fabric", "__default__", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "operation": "ADD", "attr": null} domain = default-domain >> 09/01/2025 01:34:49.535 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 09/01/2025 01:34:49.535 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 09/01/2025 01:34:49.535 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 09/01/2025 01:34:49.535 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 09/01/2025 01:34:49.535 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 09/01/2025 01:34:49.543 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-f5f3f399-f2c9-4c0e-bf35-a247e57d5409', 'type': 'global_system_config', 'uuid': '70eaa5ad-608f-40c9-a193-2be391842efd'} 09/01/2025 01:34:49.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:50.095 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:default-project:__no_rule__:ingress-access-control-list 09/01/2025 01:34:50.111 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "__no_rule__", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:50.180 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:50.159650', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.159650', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11045743276623027148, 'uuid_mslong': 8926831027094113933}}, 'parent_type': 'security-group', 'parent_uuid': '6252cda6-961a-4a8d-bbea-9dae8e162844', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '7be27985-d3e1-4e8d-994a-5cfcbb465bcc'}, 'oper': 'CREATE', 'request-id': 'req-e950410c-7673-4de8-a6eb-e40b164cf046', 'type': 'access_control_list', 'uuid': '7be27985-d3e1-4e8d-994a-5cfcbb465bcc'} 09/01/2025 01:34:50.199 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:50.201 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:50.202 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:50.203 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:50.206 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'opencontrail'] not found in DB: Unknown id: service_appliance_set default-global-system-config:opencontrail 09/01/2025 01:34:50.206 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_appliance_set identifier_name = default-global-system-config:opencontrail url = http://10.0.0.50:8082/service-appliance-sets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-appliance-set": {"uuid": null, "fq_name": ["default-global-system-config", "opencontrail"], "parent_type": "global-system-config", "service_appliance_driver": "svc_monitor.services.loadbalancer.drivers.ha_proxy.driver.OpencontrailLoadbalancerDriver", "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 5, "share": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "opencontrail"}} domain = default-domain >> 09/01/2025 01:34:50.234 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'opencontrail'], 'obj_dict': {'display_name': 'opencontrail', 'fq_name': ['default-global-system-config', 'opencontrail'], 'id_perms': {'created': '2025-09-01T01:34:50.217316', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.217316', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9255056023027074790, 'uuid_mslong': 4828196812085611406}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service_appliance_driver': 'svc_monitor.services.loadbalancer.drivers.ha_proxy.driver.OpencontrailLoadbalancerDriver', 'uuid': '4301336b-7209-4b8e-8070-906ae4de46e6'}, 'oper': 'CREATE', 'request-id': 'req-4c46b428-c5ba-4806-97a0-f574e47778f6', 'type': 'service_appliance_set', 'uuid': '4301336b-7209-4b8e-8070-906ae4de46e6'} 09/01/2025 01:34:50.241 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:default-project:__no_rule__:egress-access-control-list 09/01/2025 01:34:50.246 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "__no_rule__", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:50.264 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:50.258233', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.258233', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11657200614135326638, 'uuid_mslong': 9396845466704036391}}, 'parent_type': 'security-group', 'parent_uuid': '6252cda6-961a-4a8d-bbea-9dae8e162844', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '82684d2a-5422-4227-a1c6-b232310067ae'}, 'oper': 'CREATE', 'request-id': 'req-93e8fa48-cd11-431e-bfb1-360c392626d7', 'type': 'access_control_list', 'uuid': '82684d2a-5422-4227-a1c6-b232310067ae'} 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:50.305 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:50.306 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:50.307 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:50.340 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000000'] not found in DB: Unknown id: route_target target:64512:8000000 09/01/2025 01:34:50.340 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000000 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000000"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000000"}} domain = default-domain >> 09/01/2025 01:34:50.355 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000000'], 'obj_dict': {'display_name': 'target:64512:8000000', 'fq_name': ['target:64512:8000000'], 'id_perms': {'created': '2025-09-01T01:34:50.348208', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.348208', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9443642844578301083, 'uuid_mslong': 15519241628924461888}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd75f6c09-7b7f-4340-830e-8f21f0bfc49b'}, 'oper': 'CREATE', 'request-id': 'req-b1c43c74-64e6-4606-a260-63a70324752a', 'type': 'route_target', 'uuid': 'd75f6c09-7b7f-4340-830e-8f21f0bfc49b'} 09/01/2025 01:34:50.358 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'native'] not found in DB: Unknown id: service_appliance_set default-global-system-config:native 09/01/2025 01:34:50.359 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_appliance_set identifier_name = default-global-system-config:native url = http://10.0.0.50:8082/service-appliance-sets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-appliance-set": {"uuid": null, "fq_name": ["default-global-system-config", "native"], "parent_type": "global-system-config", "service_appliance_driver": "svc_monitor.services.loadbalancer.drivers.native.driver.OpencontrailLoadbalancerDriver", "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 5, "share": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "native"}} domain = default-domain >> 09/01/2025 01:34:50.364 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9adf9a88-a20f-4014-9216-9271c7fbc2cd object_type = routing_instance identifier_name = default-domain:default-project:dci-network:__default__ url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "9adf9a88-a20f-4014-9216-9271c7fbc2cd", "ref-type": "route-target", "ref-uuid": "d75f6c09-7b7f-4340-830e-8f21f0bfc49b", "ref-fq-name": ["target:64512:8000000"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:50.397 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', '__default__'], 'oper': 'UPDATE', 'request-id': 'req-1c6ee63b-6562-4844-bcda-41b5d0b490c7', 'type': 'routing_instance', 'uuid': '9adf9a88-a20f-4014-9216-9271c7fbc2cd'} 09/01/2025 01:34:50.403 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'native'], 'obj_dict': {'display_name': 'native', 'fq_name': ['default-global-system-config', 'native'], 'id_perms': {'created': '2025-09-01T01:34:50.382359', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.382359', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12501828825678448501, 'uuid_mslong': 14886231608596975126}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 5, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service_appliance_driver': 'svc_monitor.services.loadbalancer.drivers.native.driver.OpencontrailLoadbalancerDriver', 'uuid': 'ce9684c7-029d-4e16-ad7f-6b14554d8775'}, 'oper': 'CREATE', 'request-id': 'req-fa600bf6-b96b-4b97-90c4-58b356955ad7', 'type': 'service_appliance_set', 'uuid': 'ce9684c7-029d-4e16-ad7f-6b14554d8775'} 09/01/2025 01:34:50.431 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000001'] not found in DB: Unknown id: route_target target:64512:8000001 09/01/2025 01:34:50.432 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000001 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000001"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000001"}} domain = default-domain >> 09/01/2025 01:34:50.453 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:50.454 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:50.458 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:50.459 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:50.460 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:50.474 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000001'], 'obj_dict': {'display_name': 'target:64512:8000001', 'fq_name': ['target:64512:8000001'], 'id_perms': {'created': '2025-09-01T01:34:50.439725', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.439725', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12735328010590043084, 'uuid_mslong': 5237348473542821498}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '48aeccb0-2274-467a-b0bc-f959d94d3bcc'}, 'oper': 'CREATE', 'request-id': 'req-81caa51d-35ea-416f-9ebc-b04e99a201ad', 'type': 'route_target', 'uuid': '48aeccb0-2274-467a-b0bc-f959d94d3bcc'} 09/01/2025 01:34:50.479 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 09/01/2025 01:34:50.479 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 09/01/2025 01:34:50.479 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 09/01/2025 01:34:50.479 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 09/01/2025 01:34:50.479 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 09/01/2025 01:34:50.516 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d6b3f077-78b9-41c2-8edc-24476595a079 object_type = routing_instance identifier_name = default-domain:default-project:dci-network:dci-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "d6b3f077-78b9-41c2-8edc-24476595a079", "ref-type": "route-target", "ref-uuid": "48aeccb0-2274-467a-b0bc-f959d94d3bcc", "ref-fq-name": ["target:64512:8000001"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:50.530 7fbb3ceb7b80 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: analytics_snmp_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:50.542 7fbb3ceb7b80 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = analytics_snmp_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/analytics-snmp-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_analytics_snmp_node.py remote_ip = 10.0.0.50 body = {"analytics-snmp-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "analytics_snmp_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:50.617 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'dci-network', 'dci-network'], 'oper': 'UPDATE', 'request-id': 'req-08c7ac2d-b3a5-4ce6-b3bd-ebea3a777a54', 'type': 'routing_instance', 'uuid': 'd6b3f077-78b9-41c2-8edc-24476595a079'} 09/01/2025 01:34:50.639 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'analytics_snmp_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:50.633650', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.633650', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13313938222487280835, 'uuid_mslong': 10506029455339834136}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '91ccea20-5552-4b18-b8c4-9c45e8d99cc3'}, 'oper': 'CREATE', 'request-id': 'req-1d6bde23-1792-4119-98b5-ddcba036cd48', 'type': 'analytics_snmp_node', 'uuid': '91ccea20-5552-4b18-b8c4-9c45e8d99cc3'} 09/01/2025 01:34:50.639 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectAnalyticsSNMPInfo:ContrailConfigTrace] map 09/01/2025 01:34:50.639 7fbb3ceb7c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:50.639 7fbb3ceb7c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { analytics_snmp_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10506029455339834136, "uuid_lslong": 13313938222487280835}, "enable": true, "created": "2025-09-01T01:34:50.633650", "last_modified": "2025-09-01T01:34:50.633650", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "91ccea20-5552-4b18-b8c4-9c45e8d99cc3" } deleted = False >> 09/01/2025 01:34:50.640 7fbb3ceb7c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:50.676 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000002'] not found in DB: Unknown id: route_target target:64512:8000002 09/01/2025 01:34:50.676 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000002 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000002"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000002"}} domain = default-domain >> 09/01/2025 01:34:50.722 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000002'], 'obj_dict': {'display_name': 'target:64512:8000002', 'fq_name': ['target:64512:8000002'], 'id_perms': {'created': '2025-09-01T01:34:50.695624', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.695624', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13391815119521764036, 'uuid_mslong': 4639850035038670638}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '40640f06-1d90-4f2e-b9d9-48e5f8ccd2c4'}, 'oper': 'CREATE', 'request-id': 'req-09971ec5-2ac5-4f7f-a349-089fea18ac53', 'type': 'route_target', 'uuid': '40640f06-1d90-4f2e-b9d9-48e5f8ccd2c4'} 09/01/2025 01:34:50.734 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = fcc5deed-0637-4790-9a98-72c1e1b3a189 object_type = routing_instance identifier_name = default-domain:default-project:default-virtual-network:default-virtual-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "fcc5deed-0637-4790-9a98-72c1e1b3a189", "ref-type": "route-target", "ref-uuid": "40640f06-1d90-4f2e-b9d9-48e5f8ccd2c4", "ref-fq-name": ["target:64512:8000002"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:50.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:50.756 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'analyzer-template'] not found in DB: Unknown id: service_template default-domain:analyzer-template 09/01/2025 01:34:50.757 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:analyzer-template url = http://10.0.0.50:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "analyzer-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": null, "service_type": "analyzer", "image_name": "analyzer", "service_scaling": false, "interface_type": [{"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}], "flavor": "m1.medium", "ordered_interfaces": true, "service_virtualization_type": "virtual-machine", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "analyzer-template"}} domain = default-domain >> 09/01/2025 01:34:50.764 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'default-virtual-network', 'default-virtual-network'], 'oper': 'UPDATE', 'request-id': 'req-7ec8d4b5-0ac2-44d8-a591-e2916f716e74', 'type': 'routing_instance', 'uuid': 'fcc5deed-0637-4790-9a98-72c1e1b3a189'} 09/01/2025 01:34:50.786 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000003'] not found in DB: Unknown id: route_target target:64512:8000003 09/01/2025 01:34:50.787 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000003 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000003"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000003"}} domain = default-domain >> 09/01/2025 01:34:50.794 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'analyzer-template'], 'obj_dict': {'display_name': 'analyzer-template', 'fq_name': ['default-domain', 'analyzer-template'], 'id_perms': {'created': '2025-09-01T01:34:50.782345', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.782345', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12858072168594902976, 'uuid_mslong': 11776431650019952100}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': [{'tenant': 'domain:ce056a4b-12ea-41a5-896d-393d506f2011', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': None, 'service_scaling': False, 'service_type': 'analyzer', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': 'a36e4a31-051b-45e4-b271-0c83296f8fc0'}, 'oper': 'CREATE', 'request-id': 'req-c275d7c6-4dcc-4c74-8863-db3540fbc449', 'type': 'service_template', 'uuid': 'a36e4a31-051b-45e4-b271-0c83296f8fc0'} 09/01/2025 01:34:50.810 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000003'], 'obj_dict': {'display_name': 'target:64512:8000003', 'fq_name': ['target:64512:8000003'], 'id_perms': {'created': '2025-09-01T01:34:50.798909', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.798909', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10497902573199109149, 'uuid_mslong': 18412618679371710727}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ff86c322-78ac-4107-91b0-0ac522312c1d'}, 'oper': 'CREATE', 'request-id': 'req-c094229e-a3f1-43ff-bc53-ecb3edf74e3d', 'type': 'route_target', 'uuid': 'ff86c322-78ac-4107-91b0-0ac522312c1d'} 09/01/2025 01:34:50.828 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'nat-template'] not found in DB: Unknown id: service_template default-domain:nat-template 09/01/2025 01:34:50.829 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:nat-template url = http://10.0.0.50:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "nat-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "firewall", "image_name": "analyzer", "service_scaling": false, "interface_type": [{"service_interface_type": "management", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "right", "shared_ip": false, "static_route_enable": false}], "flavor": "m1.medium", "ordered_interfaces": true, "service_virtualization_type": "virtual-machine", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "nat-template"}} domain = default-domain >> 09/01/2025 01:34:50.860 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 23054292-38bc-4b6f-b5fa-0f8021a7362f object_type = routing_instance identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "23054292-38bc-4b6f-b5fa-0f8021a7362f", "ref-type": "route-target", "ref-uuid": "ff86c322-78ac-4107-91b0-0ac522312c1d", "ref-fq-name": ["target:64512:8000003"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:50.866 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'nat-template'], 'obj_dict': {'display_name': 'nat-template', 'fq_name': ['default-domain', 'nat-template'], 'id_perms': {'created': '2025-09-01T01:34:50.858040', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.858040', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11607138525345186793, 'uuid_mslong': 17820619935228381262}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': [{'tenant': 'domain:ce056a4b-12ea-41a5-896d-393d506f2011', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': 'm1.medium', 'image_name': 'analyzer', 'instance_data': None, 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'virtual-machine', 'version': 1, 'vrouter_instance_type': None}, 'uuid': 'f74f8f69-d488-484e-a114-d6fdd5f383e9'}, 'oper': 'CREATE', 'request-id': 'req-0376e46a-a073-41a6-aac8-bbb826241801', 'type': 'service_template', 'uuid': 'f74f8f69-d488-484e-a114-d6fdd5f383e9'} 09/01/2025 01:34:50.879 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'netns-snat-template'] not found in DB: Unknown id: service_template default-domain:netns-snat-template 09/01/2025 01:34:50.879 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:netns-snat-template url = http://10.0.0.50:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "netns-snat-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "source-nat", "image_name": null, "service_scaling": true, "interface_type": [{"service_interface_type": "right", "shared_ip": true, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": true, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "network-namespace", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "netns-snat-template"}} domain = default-domain >> 09/01/2025 01:34:50.886 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-790d1dcd-769c-4e4d-96ae-5c52e3ae74e4', 'type': 'routing_instance', 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'} 09/01/2025 01:34:50.912 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'netns-snat-template'], 'obj_dict': {'display_name': 'netns-snat-template', 'fq_name': ['default-domain', 'netns-snat-template'], 'id_perms': {'created': '2025-09-01T01:34:50.906086', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:50.906086', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13473599966090706278, 'uuid_mslong': 13884075901440511548}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': [{'tenant': 'domain:ce056a4b-12ea-41a5-896d-393d506f2011', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'source-nat', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': 'c0ae2578-abec-4e3c-bafb-d7c8708d3d66'}, 'oper': 'CREATE', 'request-id': 'req-213a08ab-288f-4ef4-8664-1adb24538bbc', 'type': 'service_template', 'uuid': 'c0ae2578-abec-4e3c-bafb-d7c8708d3d66'} 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:50.943 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:50.944 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:50.945 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:50.953 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'haproxy-loadbalancer-template'] not found in DB: Unknown id: service_template default-domain:haproxy-loadbalancer-template 09/01/2025 01:34:50.954 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:haproxy-loadbalancer-template url = http://10.0.0.50:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "haproxy-loadbalancer-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "in-network-nat", "service_type": "loadbalancer", "image_name": null, "service_scaling": true, "interface_type": [{"service_interface_type": "right", "shared_ip": true, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": true, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "network-namespace", "availability_zone_enable": false, "vrouter_instance_type": null, "instance_data": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "haproxy-loadbalancer-template"}} domain = default-domain >> 09/01/2025 01:34:50.954 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'] not found in DB: Unknown id: access_control_list default-domain:default-project:ip-fabric:ip-fabric 09/01/2025 01:34:50.958 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "parent_type": "virtual-network", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ip-fabric"}} domain = default-domain >> 09/01/2025 01:34:51.011 7fbb3ceb7040 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: database_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:51.019 7fbb3ceb7040 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = database_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/database-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_database_node.py remote_ip = 10.0.0.50 body = {"database-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "database_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:51.024 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'obj_dict': {'display_name': 'haproxy-loadbalancer-template', 'fq_name': ['default-domain', 'haproxy-loadbalancer-template'], 'id_perms': {'created': '2025-09-01T01:34:51.014898', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:51.014898', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10967202183717175702, 'uuid_mslong': 16936036994159822975}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': [{'tenant': 'domain:ce056a4b-12ea-41a5-896d-393d506f2011', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': None, 'instance_data': None, 'interface_type': [{'service_interface_type': 'right', 'shared_ip': True, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': True, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'in-network-nat', 'service_scaling': True, 'service_type': 'loadbalancer', 'service_virtualization_type': 'network-namespace', 'version': 1, 'vrouter_instance_type': None}, 'uuid': 'eb08e3ea-4bc5-487f-9833-54478459c196'}, 'oper': 'CREATE', 'request-id': 'req-7f851d7d-963e-4457-86f5-f73ba73a7596', 'type': 'service_template', 'uuid': 'eb08e3ea-4bc5-487f-9833-54478459c196'} 09/01/2025 01:34:51.077 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'deny'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000002'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'ip-fabric', 'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'id_perms': {'created': '2025-09-01T01:34:51.020758', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:51.020758', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12041321501398024398, 'uuid_mslong': 12140204589544197638}}, 'parent_type': 'virtual-network', 'parent_uuid': 'e07e6e04-4e93-4043-ab83-32b8f8854bbe', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce'}, 'oper': 'CREATE', 'request-id': 'req-a8ad174b-a10d-409e-ba51-8759717bd066', 'type': 'access_control_list', 'uuid': 'a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce'} 09/01/2025 01:34:51.085 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'docker-template'] not found in DB: Unknown id: service_template default-domain:docker-template 09/01/2025 01:34:51.085 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = service_template identifier_name = default-domain:docker-template url = http://10.0.0.50:8082/service-templates operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"service-template": {"uuid": null, "fq_name": ["default-domain", "docker-template"], "parent_type": "domain", "service_template_properties": {"version": 1, "service_mode": "transparent", "service_type": "firewall", "image_name": "ubuntu", "service_scaling": false, "interface_type": [{"service_interface_type": "management", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "left", "shared_ip": false, "static_route_enable": false}, {"service_interface_type": "right", "shared_ip": false, "static_route_enable": false}], "flavor": null, "ordered_interfaces": true, "service_virtualization_type": "vrouter-instance", "availability_zone_enable": false, "vrouter_instance_type": "docker", "instance_data": "{\"command\":\"/bin/bash\"}"}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "docker-template"}} domain = default-domain >> 09/01/2025 01:34:51.109 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 7be27985-d3e1-4e8d-994a-5cfcbb465bcc object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:ingress-access-control-list url = http://10.0.0.50:8082/access-control-list/7be27985-d3e1-4e8d-994a-5cfcbb465bcc operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "7be27985-d3e1-4e8d-994a-5cfcbb465bcc", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "access_control_list_hash": 3520976145078303111}} domain = default-domain >> 09/01/2025 01:34:51.111 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'database_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:51.091221', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:51.091221', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11598019388747453457, 'uuid_mslong': 11692571017731722543}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a2445b64-06ed-4d2f-a0f4-712f64629411'}, 'oper': 'CREATE', 'request-id': 'req-83260608-ad4f-478a-a182-7be52f077295', 'type': 'database_node', 'uuid': 'a2445b64-06ed-4d2f-a0f4-712f64629411'} 09/01/2025 01:34:51.113 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectDatabaseInfo:ContrailConfigTrace] map 09/01/2025 01:34:51.113 7fbb3ceb7040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:51.113 7fbb3ceb7040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { database_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11692571017731722543, "uuid_lslong": 11598019388747453457}, "enable": true, "created": "2025-09-01T01:34:51.091221", "last_modified": "2025-09-01T01:34:51.091221", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "a2445b64-06ed-4d2f-a0f4-712f64629411" } deleted = False >> 09/01/2025 01:34:51.113 7fbb3ceb7040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:51.133 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'docker-template'], 'obj_dict': {'display_name': 'docker-template', 'fq_name': ['default-domain', 'docker-template'], 'id_perms': {'created': '2025-09-01T01:34:51.126984', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:51.126984', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12259230669281774089, 'uuid_mslong': 9795553576050508563}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': [{'tenant': 'domain:ce056a4b-12ea-41a5-896d-393d506f2011', 'tenant_access': 5}]}, 'service_template_properties': {'availability_zone_enable': False, 'flavor': None, 'image_name': 'ubuntu', 'instance_data': '{"command":"/bin/bash"}', 'interface_type': [{'service_interface_type': 'management', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'left', 'shared_ip': False, 'static_route_enable': False}, {'service_interface_type': 'right', 'shared_ip': False, 'static_route_enable': False}], 'ordered_interfaces': True, 'service_mode': 'transparent', 'service_scaling': False, 'service_type': 'firewall', 'service_virtualization_type': 'vrouter-instance', 'version': 1, 'vrouter_instance_type': 'docker'}, 'uuid': '87f0cc14-0ecc-4313-aa21-89576e1d4a09'}, 'oper': 'CREATE', 'request-id': 'req-c077d094-eca3-4f1c-aa1d-8f898a4dd81c', 'type': 'service_template', 'uuid': '87f0cc14-0ecc-4313-aa21-89576e1d4a09'} 09/01/2025 01:34:51.141 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-b7851b79-6e7b-41fa-8315-d3774588e2ae', 'type': 'access_control_list', 'uuid': '7be27985-d3e1-4e8d-994a-5cfcbb465bcc'} 09/01/2025 01:34:51.192 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 82684d2a-5422-4227-a1c6-b232310067ae object_type = access_control_list identifier_name = default-domain:default-project:__no_rule__:egress-access-control-list url = http://10.0.0.50:8082/access-control-list/82684d2a-5422-4227-a1c6-b232310067ae operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "82684d2a-5422-4227-a1c6-b232310067ae", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "access_control_list_hash": 3520976145078303111}} domain = default-domain >> 09/01/2025 01:34:51.216 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', '__no_rule__', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-a9af179c-4117-4193-9626-b5ef5a743d03', 'type': 'access_control_list', 'uuid': '82684d2a-5422-4227-a1c6-b232310067ae'} 09/01/2025 01:34:51.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:52.023 7fbb3ceb7360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:34:52.027 7fbb3ceb7360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:34:52.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:53.180 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:53.181 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:34:53.182 7fbb3ceb7860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:53.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:54.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:54.848 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:54.853 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:54.854 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:54.855 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:54.856 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:54.916 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: config_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:54.925 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = config_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/config-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_config_node.py remote_ip = 10.0.0.50 body = {"config-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "config_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:54.948 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'config_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:54.934849', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:54.934849', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11050905052042072441, 'uuid_mslong': 16245146586780484127}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1725ac0-0d33-421f-995c-b39839899179'}, 'oper': 'CREATE', 'request-id': 'req-77d95627-86ba-4107-afa8-8d753536bb23', 'type': 'config_node', 'uuid': 'e1725ac0-0d33-421f-995c-b39839899179'} 09/01/2025 01:34:54.948 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigNode:ContrailConfigTrace] map 09/01/2025 01:34:54.948 7fbb3ceb7860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:54.949 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { config_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16245146586780484127, "uuid_lslong": 11050905052042072441}, "enable": true, "created": "2025-09-01T01:34:54.934849", "last_modified": "2025-09-01T01:34:54.934849", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e1725ac0-0d33-421f-995c-b39839899179" } deleted = False >> 09/01/2025 01:34:54.949 7fbb3ceb7860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.374 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_domain_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource domain: 'parent_type' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field projects in resource domain: 'projects' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespaces in resource domain: 'namespaces' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_templates in resource domain: 'service_templates' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNSs in resource domain: 'virtual_DNSs' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource domain: 'api_access_lists' 09/01/2025 01:34:55.376 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource domain: 'tag_refs' 09/01/2025 01:34:55.379 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default'] not found in DB: Unknown id: project default-domain:k8s-default 09/01/2025 01:34:55.380 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = project identifier_name = default-domain:k8s-default url = http://10.0.0.50:8082/projects operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"project": {"uuid": null, "fq_name": ["default-domain", "k8s-default"], "parent_type": "domain", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default"}} domain = default-domain >> 09/01/2025 01:34:55.392 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'obj_dict': {'display_name': 'k8s-default', 'fq_name': ['default-domain', 'k8s-default'], 'id_perms': {'created': '2025-09-01T01:34:55.387855', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.387855', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11357663924834040824, 'uuid_mslong': 10386380444838743678}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'}, 'oper': 'CREATE', 'request-id': 'req-bd191f7e-5bc6-4e0b-b0dc-2d406720360c', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:34:55.392 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:34:55.393 7fbb3ceb79a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.393 7fbb3ceb79a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:34:55.387855", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:34:55.393 7fbb3ceb79a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.399 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'default-application-policy-set'] not found in DB: Unknown id: application_policy_set default-domain:k8s-default:default-application-policy-set 09/01/2025 01:34:55.404 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-domain:k8s-default:default-application-policy-set url = http://127.0.0.1/application-policy-sets operation = post body = {"application-policy-set": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "default-application-policy-set"], "parent_type": "project", "all_applications": true, "display_name": "default-application-policy-set", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:34:55.427 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'display_name': 'default-application-policy-set', 'fq_name': ['default-domain', 'k8s-default', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:34:55.418880', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.418880', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12899231214931829714, 'uuid_mslong': 17526535842386496605}}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'f33ac37d-3499-4c5d-b303-467269a463d2'}, 'oper': 'CREATE', 'request-id': 'req-bd191f7e-5bc6-4e0b-b0dc-2d406720360c', 'type': 'application_policy_set', 'uuid': 'f33ac37d-3499-4c5d-b303-467269a463d2'} 09/01/2025 01:34:55.427 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:34:55.427 7fbb3ceb7ea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.428 7fbb3ceb7ea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:default-application-policy-set elements = { all_applications : truedisplay_name : "default-application-policy-set"fq_name : ["default-domain", "k8s-default", "default-application-policy-set"]id_perms : {"permissions": {"owner": "contrail-api", "owner_access": 7, "group": "admin", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17526535842386496605, "uuid_lslong": 12899231214931829714}, "enable": true, "created": "2025-09-01T01:34:55.418880", "last_modified": "2025-09-01T01:34:55.418880", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "f33ac37d-3499-4c5d-b303-467269a463d2" } deleted = False >> 09/01/2025 01:34:55.428 7fbb3ceb7ea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.438 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:34:55.451 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-bd191f7e-5bc6-4e0b-b0dc-2d406720360c', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:34:55.456 7fbb41e2d220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.457 7fbb41e2d220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:34:55.442376", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:34:55.457 7fbb41e2d220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.465 7fbb3d0e3ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'default'] not found in DB: Unknown id: security_group default-domain:k8s-default:default 09/01/2025 01:34:55.466 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-default:default url = http://cn-jenkins-deploy-platform-juju-k8s-1454-1:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-api remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "default"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "a9777858-b10b-4f90-ae9a-ee86bb4e8bb1", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-default:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "a0ce129a-72d1-4990-8cd9-2cadac31ddea", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-default:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "f588a086-f096-477b-bc3b-6a0e980ef7ed", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "332d697a-4608-42c5-bfb1-5aa301238183", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "default"}} domain = default-domain >> 09/01/2025 01:34:55.533 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'default'], 'obj_dict': {'display_name': 'default', 'fq_name': ['default-domain', 'k8s-default', 'default'], 'id_perms': {'created': '2025-09-01T01:34:55.484361', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:34:55.484361', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11936403218885516667, 'uuid_mslong': 6540726343857162531}}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'a9777858-b10b-4f90-ae9a-ee86bb4e8bb1', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-default:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'a0ce129a-72d1-4990-8cd9-2cadac31ddea', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-default:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'f588a086-f096-477b-bc3b-6a0e980ef7ed', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '332d697a-4608-42c5-bfb1-5aa301238183', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000002, 'uuid': '5ac553f4-9aa5-4923-a5a6-9f7c5d801d7b'}, 'oper': 'CREATE', 'request-id': 'req-87bead24-6f46-492a-b835-8fdf789d0ce8', 'type': 'security_group', 'uuid': '5ac553f4-9aa5-4923-a5a6-9f7c5d801d7b'} 09/01/2025 01:34:55.553 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'default', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:default:ingress-access-control-list 09/01/2025 01:34:55.556 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:default:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "default", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000002", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "a9777858-b10b-4f90-ae9a-ee86bb4e8bb1", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000002", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "a0ce129a-72d1-4990-8cd9-2cadac31ddea", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:55.574 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-default-sg'] not found in DB: Unknown id: security_group default-domain:k8s-default:k8s-default-default-sg 09/01/2025 01:34:55.609 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-default:k8s-default-default-sg url = http://10.0.0.50:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-default-sg"}} domain = default-domain >> 09/01/2025 01:34:55.634 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'default', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000002', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'a9777858-b10b-4f90-ae9a-ee86bb4e8bb1'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000002', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'a0ce129a-72d1-4990-8cd9-2cadac31ddea'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-default', 'default', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:55.565399', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.565399', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12393600091174732163, 'uuid_mslong': 5435922950334663122}}, 'parent_type': 'security-group', 'parent_uuid': '5ac553f4-9aa5-4923-a5a6-9f7c5d801d7b', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'uuid': '4b70471f-6117-49d2-abfe-e99e6ab80183'}, 'oper': 'CREATE', 'request-id': 'req-1aaa4255-fd97-4aa1-8586-a82862deafbe', 'type': 'access_control_list', 'uuid': '4b70471f-6117-49d2-abfe-e99e6ab80183'} 09/01/2025 01:34:55.644 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'default', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:default:egress-access-control-list 09/01/2025 01:34:55.647 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:default:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "default", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "f588a086-f096-477b-bc3b-6a0e980ef7ed", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "332d697a-4608-42c5-bfb1-5aa301238183", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:55.653 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg'], 'obj_dict': {'display_name': 'k8s-default-default-sg', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg'], 'id_perms': {'created': '2025-09-01T01:34:55.649202', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:34:55.649202', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13595223614054563807, 'uuid_mslong': 5613512272088154649}}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': []}, 'security_group_id': 8000003, 'uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf'}, 'oper': 'CREATE', 'request-id': 'req-862169ea-8f04-4eb4-b8a9-323bb8eb9c3a', 'type': 'security_group', 'uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf'} 09/01/2025 01:34:55.733 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'default', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'f588a086-f096-477b-bc3b-6a0e980ef7ed'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '332d697a-4608-42c5-bfb1-5aa301238183'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-default', 'default', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:55.667490', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.667490', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13388604008182575585, 'uuid_mslong': 7374738075550172945}}, 'parent_type': 'security-group', 'parent_uuid': '5ac553f4-9aa5-4923-a5a6-9f7c5d801d7b', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'uuid': '66585534-e8b7-4711-b9cd-e068da17f1e1'}, 'oper': 'CREATE', 'request-id': 'req-dc2a2ba5-0558-47fa-ae44-cc96394be8fc', 'type': 'access_control_list', 'uuid': '66585534-e8b7-4711-b9cd-e068da17f1e1'} 09/01/2025 01:34:55.737 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: chown: 4de733ab-781c-4219-bcab-efd8adb97fdf owner set to 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 09/01/2025 01:34:55.742 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-8e1185cb-738d-413f-9236-17f30c42c65b', 'type': 'security_group', 'uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf'} 09/01/2025 01:34:55.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:55.807 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:k8s-default-default-sg:ingress-access-control-list 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:34:55.808 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:34:55.809 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:34:55.811 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-default-sg", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:55.813 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s'] not found in DB: Unknown id: application_policy_set default-policy-management:k8s 09/01/2025 01:34:55.815 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/application-policy-sets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"application-policy-set": {"uuid": null, "fq_name": ["default-policy-management", "k8s"], "parent_type": "policy-management", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s"}} domain = default-domain >> 09/01/2025 01:34:55.831 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:55.820914', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.820914', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10221026790512302696, 'uuid_mslong': 724960809861532916}}, 'parent_type': 'security-group', 'parent_uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'uuid': '0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68'}, 'oper': 'CREATE', 'request-id': 'req-54de4dcc-5324-4370-98b0-d85f92399627', 'type': 'access_control_list', 'uuid': '0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68'} 09/01/2025 01:34:55.839 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:k8s-default-default-sg:egress-access-control-list 09/01/2025 01:34:55.843 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'obj_dict': {'display_name': 'k8s', 'fq_name': ['default-policy-management', 'k8s'], 'id_perms': {'created': '2025-09-01T01:34:55.834723', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.834723', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12603312532585330301, 'uuid_mslong': 1917128780955733221}}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'}, 'oper': 'CREATE', 'request-id': 'req-a0620321-560d-4dab-b586-65ca6b69f87f', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:55.843 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:34:55.844 7fbb3ceb7c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.844 7fbb3ceb7c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:55.834723", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:55.844 7fbb3ceb7c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.845 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-default-sg", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:55.868 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:55.857136', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.857136', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10774130541549910497, 'uuid_mslong': 16845570577868801345}}, 'parent_type': 'security-group', 'parent_uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'uuid': 'e9c77d30-5ca4-4541-9585-66a4ccc4f9e1'}, 'oper': 'CREATE', 'request-id': 'req-7f692d85-96b4-4f98-8275-a07a203ce93c', 'type': 'access_control_list', 'uuid': 'e9c77d30-5ca4-4541-9585-66a4ccc4f9e1'} 09/01/2025 01:34:55.886 7fbb3ceb7ea0 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:34:55.888 7fbb3ceb7ea0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:34:55.902 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 09/01/2025 01:34:55.902 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 09/01/2025 01:34:55.902 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 09/01/2025 01:34:55.907 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 87a687b0-9781-4994-aa87-dbc3f38997c0 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = http://10.0.0.50:8082/global-vrouter-config/87a687b0-9781-4994-aa87-dbc3f38997c0 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"global-vrouter-config": {"uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0", "port_translation_pools": {"port_translation_pool": [{"protocol": "tcp", "port_range": {"start_port": 56000, "end_port": 57023}, "port_count": null}, {"protocol": "udp", "port_range": {"start_port": 57024, "end_port": 58047}, "port_count": null}]}}} domain = default-domain >> 09/01/2025 01:34:55.918 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-eef11707-29e3-4fad-9cd7-903ad0babe99', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:34:55.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-network: 'network_ipam_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_refs in resource virtual-network: 'network_policy_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:34:55.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:34:55.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:34:55.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:34:55.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:34:55.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:34:55.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system'] not found in DB: Unknown id: project default-domain:k8s-kube-system 09/01/2025 01:34:55.977 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = project identifier_name = default-domain:k8s-kube-system url = http://10.0.0.50:8082/projects operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"project": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system"], "parent_type": "domain", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-system"}} domain = default-domain >> 09/01/2025 01:34:55.990 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'obj_dict': {'display_name': 'k8s-kube-system', 'fq_name': ['default-domain', 'k8s-kube-system'], 'id_perms': {'created': '2025-09-01T01:34:55.985402', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:55.985402', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10013454790172241632, 'uuid_mslong': 11951427749879825585}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'}, 'oper': 'CREATE', 'request-id': 'req-41dd65dc-f795-45e5-a745-6d1962e85f36', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:34:55.990 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:34:55.991 7fbb41e2d180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.991 7fbb41e2d180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:34:55.985402", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:34:55.991 7fbb41e2d180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:55.995 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'default-application-policy-set'] not found in DB: Unknown id: application_policy_set default-domain:k8s-kube-system:default-application-policy-set 09/01/2025 01:34:55.999 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-domain:k8s-kube-system:default-application-policy-set url = http://127.0.0.1/application-policy-sets operation = post body = {"application-policy-set": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "parent_type": "project", "all_applications": true, "display_name": "default-application-policy-set", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:34:56.015 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'display_name': 'default-application-policy-set', 'fq_name': ['default-domain', 'k8s-kube-system', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:34:56.012086', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.012086', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10948140934525109059, 'uuid_mslong': 17021587060483244415}}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'ec38d33f-4d7d-417f-97ef-9c2c99484343'}, 'oper': 'CREATE', 'request-id': 'req-41dd65dc-f795-45e5-a745-6d1962e85f36', 'type': 'application_policy_set', 'uuid': 'ec38d33f-4d7d-417f-97ef-9c2c99484343'} 09/01/2025 01:34:56.016 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:34:56.016 7fbb3ceb7220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.016 7fbb3ceb7220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:default-application-policy-set elements = { all_applications : truedisplay_name : "default-application-policy-set"fq_name : ["default-domain", "k8s-kube-system", "default-application-policy-set"]id_perms : {"permissions": {"owner": "contrail-api", "owner_access": 7, "group": "admin", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17021587060483244415, "uuid_lslong": 10948140934525109059}, "enable": true, "created": "2025-09-01T01:34:56.012086", "last_modified": "2025-09-01T01:34:56.012086", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "ec38d33f-4d7d-417f-97ef-9c2c99484343" } deleted = False >> 09/01/2025 01:34:56.016 7fbb3ceb7220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.030 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:34:56.045 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-41dd65dc-f795-45e5-a745-6d1962e85f36', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:34:56.052 7fbb3ceb79a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.052 7fbb3ceb79a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:34:56.034109", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:34:56.052 7fbb3ceb79a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.055 7fbb3d0e3ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'default'] not found in DB: Unknown id: security_group default-domain:k8s-kube-system:default 09/01/2025 01:34:56.055 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-system:default url = http://cn-jenkins-deploy-platform-juju-k8s-1454-1:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-api remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "default"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "0343e44d-b0e1-4539-8c19-d2156da9cc62", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-system:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "a92a8a05-b289-4b87-89c3-98c0865faf30", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-system:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "9cd8ca77-a919-448f-88f9-6db9c556a8f1", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "5f74e63a-1bb1-40ad-9d17-b59bdd9d5719", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "default"}} domain = default-domain >> 09/01/2025 01:34:56.288 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'default'], 'obj_dict': {'display_name': 'default', 'fq_name': ['default-domain', 'k8s-kube-system', 'default'], 'id_perms': {'created': '2025-09-01T01:34:56.235704', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:34:56.235704', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11024785080048304205, 'uuid_mslong': 14009118518343257964}}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '0343e44d-b0e1-4539-8c19-d2156da9cc62', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-system:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'a92a8a05-b289-4b87-89c3-98c0865faf30', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-system:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '9cd8ca77-a919-448f-88f9-6db9c556a8f1', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '5f74e63a-1bb1-40ad-9d17-b59bdd9d5719', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000004, 'uuid': 'c26a6311-a632-4f6c-98ff-e79e554e404d'}, 'oper': 'CREATE', 'request-id': 'req-4a0481de-ef18-4672-b4dd-de9aaa349670', 'type': 'security_group', 'uuid': 'c26a6311-a632-4f6c-98ff-e79e554e404d'} 09/01/2025 01:34:56.307 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'] not found in DB: Unknown id: security_group default-domain:k8s-kube-system:k8s-kube-system-default-sg 09/01/2025 01:34:56.312 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg url = http://10.0.0.50:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-system-default-sg"}} domain = default-domain >> 09/01/2025 01:34:56.334 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'default', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-system:default:ingress-access-control-list 09/01/2025 01:34:56.339 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-system:default:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "default", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000004", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "0343e44d-b0e1-4539-8c19-d2156da9cc62", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000004", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "a92a8a05-b289-4b87-89c3-98c0865faf30", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:56.350 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'], 'obj_dict': {'display_name': 'k8s-kube-system-default-sg', 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'], 'id_perms': {'created': '2025-09-01T01:34:56.343303', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:34:56.343303', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11198606753506992789, 'uuid_mslong': 16211337997743114230}}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': []}, 'security_group_id': 8000005, 'uuid': 'e0fa3e04-8145-47f6-9b69-718272132295'}, 'oper': 'CREATE', 'request-id': 'req-ccd58c30-7b81-4365-9b61-97e54a8c03ce', 'type': 'security_group', 'uuid': 'e0fa3e04-8145-47f6-9b69-718272132295'} 09/01/2025 01:34:56.426 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'default', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000004', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '0343e44d-b0e1-4539-8c19-d2156da9cc62'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000004', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'a92a8a05-b289-4b87-89c3-98c0865faf30'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-system', 'default', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:56.357752', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.357752', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13076623060756524689, 'uuid_mslong': 3924770634606397527}}, 'parent_type': 'security-group', 'parent_uuid': 'c26a6311-a632-4f6c-98ff-e79e554e404d', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'uuid': '36779619-0c99-4857-b579-7f640e12ee91'}, 'oper': 'CREATE', 'request-id': 'req-88880579-f6ed-40c8-8e6d-21e041eb2eac', 'type': 'access_control_list', 'uuid': '36779619-0c99-4857-b579-7f640e12ee91'} 09/01/2025 01:34:56.431 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: chown: e0fa3e04-8145-47f6-9b69-718272132295 owner set to a5dc0037-9655-44b1-8af6-f013fea66ee0 09/01/2025 01:34:56.436 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-240070e3-47f5-4369-b66e-571fc8a75cef', 'type': 'security_group', 'uuid': 'e0fa3e04-8145-47f6-9b69-718272132295'} 09/01/2025 01:34:56.437 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'default', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-system:default:egress-access-control-list 09/01/2025 01:34:56.445 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-system:default:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "default", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "9cd8ca77-a919-448f-88f9-6db9c556a8f1", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "5f74e63a-1bb1-40ad-9d17-b59bdd9d5719", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:56.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-ip-fabric-ipam'] not found in DB: Unknown id: network_ipam default-domain:k8s-default:k8s-ip-fabric-ipam 09/01/2025 01:34:56.453 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_ipam identifier_name = default-domain:k8s-default:k8s-ip-fabric-ipam url = http://10.0.0.50:8082/network-ipams operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-ipam": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-ip-fabric-ipam"], "parent_type": "project", "ipam_subnets": {"subnets": [{"subnet": {"ip_prefix": "10.64.0.0", "ip_prefix_len": 12}, "default_gateway": null, "dns_server_address": null, "subnet_uuid": null, "enable_dhcp": true, "dns_nameservers": [], "allocation_pools": [], "addr_from_start": null, "dhcp_option_list": null, "host_routes": null, "subnet_name": null, "alloc_unit": 1, "created": null, "last_modified": null, "subscriber_tag": null, "vlan_tag": null, "dhcp_relay_server": []}]}, "ipam_subnet_method": "flat-subnet", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-ip-fabric-ipam"}} domain = default-domain >> 09/01/2025 01:34:56.507 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'default', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '9cd8ca77-a919-448f-88f9-6db9c556a8f1'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '5f74e63a-1bb1-40ad-9d17-b59bdd9d5719'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-system', 'default', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:56.457597', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.457597', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11506460743523033639, 'uuid_mslong': 6215132166285246947}}, 'parent_type': 'security-group', 'parent_uuid': 'c26a6311-a632-4f6c-98ff-e79e554e404d', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'uuid': '564095c6-ab89-41e3-9faf-291517aae227'}, 'oper': 'CREATE', 'request-id': 'req-ac8f95ef-cfd2-4c90-b51d-d56b104064e6', 'type': 'access_control_list', 'uuid': '564095c6-ab89-41e3-9faf-291517aae227'} 09/01/2025 01:34:56.526 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-ip-fabric-ipam'], 'obj_dict': {'display_name': 'k8s-ip-fabric-ipam', 'fq_name': ['default-domain', 'k8s-default', 'k8s-ip-fabric-ipam'], 'id_perms': {'created': '2025-09-01T01:34:56.516615', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.516615', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12876161066528670497, 'uuid_mslong': 14866473943478387208}}, 'ipam_subnet_method': 'flat-subnet', 'ipam_subnets': {'subnets': [{'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': '10.79.255.254', 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': '10.79.255.253', 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '10.64.0.0', 'ip_prefix_len': 12}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'ce505349-1fa1-4208-b2b1-5044e9280321'}, 'oper': 'CREATE', 'request-id': 'req-91f7aff8-08b6-4a47-bbf5-4166f8477cc5', 'type': 'network_ipam', 'uuid': 'ce505349-1fa1-4208-b2b1-5044e9280321'} 09/01/2025 01:34:56.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-pod-ipam'] not found in DB: Unknown id: network_ipam default-domain:k8s-default:k8s-pod-ipam 09/01/2025 01:34:56.540 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_ipam identifier_name = default-domain:k8s-default:k8s-pod-ipam url = http://10.0.0.50:8082/network-ipams operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-ipam": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-pod-ipam"], "parent_type": "project", "ipam_subnets": {"subnets": [{"subnet": {"ip_prefix": "10.32.0.0", "ip_prefix_len": 12}, "default_gateway": null, "dns_server_address": null, "subnet_uuid": null, "enable_dhcp": true, "dns_nameservers": [], "allocation_pools": [], "addr_from_start": null, "dhcp_option_list": null, "host_routes": null, "subnet_name": null, "alloc_unit": 1, "created": null, "last_modified": null, "subscriber_tag": null, "vlan_tag": null, "dhcp_relay_server": []}]}, "ipam_subnet_method": "flat-subnet", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-pod-ipam"}} domain = default-domain >> 09/01/2025 01:34:56.558 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-system:k8s-kube-system-default-sg:ingress-access-control-list 09/01/2025 01:34:56.564 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-pod-ipam'], 'obj_dict': {'display_name': 'k8s-pod-ipam', 'fq_name': ['default-domain', 'k8s-default', 'k8s-pod-ipam'], 'id_perms': {'created': '2025-09-01T01:34:56.554258', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.554258', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10520340932493082396, 'uuid_mslong': 5725016635900381083}}, 'ipam_subnet_method': 'flat-subnet', 'ipam_subnets': {'subnets': [{'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': '10.47.255.254', 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': '10.47.255.253', 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '10.32.0.0', 'ip_prefix_len': 12}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '4f73584c-440b-479b-91ff-c256c5407f1c'}, 'oper': 'CREATE', 'request-id': 'req-407f4093-370d-4446-804c-7616fae2fc43', 'type': 'network_ipam', 'uuid': '4f73584c-440b-479b-91ff-c256c5407f1c'} 09/01/2025 01:34:56.571 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:56.589 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:56.581158', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.581158', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9762737177291075290, 'uuid_mslong': 2998260824958910946}}, 'parent_type': 'security-group', 'parent_uuid': 'e0fa3e04-8145-47f6-9b69-718272132295', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'uuid': '299bf655-b7a5-41e2-877c-35bccfef2eda'}, 'oper': 'CREATE', 'request-id': 'req-9f39a7cb-d7d5-4f96-bf25-7339bd7a5d2f', 'type': 'access_control_list', 'uuid': '299bf655-b7a5-41e2-877c-35bccfef2eda'} 09/01/2025 01:34:56.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-pod-network'] not found in DB: Unknown id: virtual_network default-domain:k8s-default:k8s-default-pod-network 09/01/2025 01:34:56.621 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-networks operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-pod-network"], "parent_type": "project", "virtual_network_properties": {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null}, "fabric_snat": true, "address_allocation_mode": "flat-subnet-only", "network_ipam_refs": [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-pod-network"}} domain = default-domain >> 09/01/2025 01:34:56.643 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-system:k8s-kube-system-default-sg:egress-access-control-list 09/01/2025 01:34:56.645 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:34:56.693 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'obj_dict': {'address_allocation_mode': 'flat-subnet-only', 'display_name': 'k8s-default-pod-network', 'fabric_snat': True, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'id_perms': {'created': '2025-09-01T01:34:56.642429', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.642429', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10518379392776784806, 'uuid_mslong': 15443089546972253873}}, 'is_shared': False, 'network_ipam_refs': [{'attr': {'host_routes': None, 'ipam_subnets': [{'subnet_uuid': 'eada5f2d-448e-4621-9fea-812925eb46e9'}]}, 'to': ['default-domain', 'k8s-default', 'k8s-pod-ipam'], 'uuid': '4f73584c-440b-479b-91ff-c256c5407f1c'}], 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6', 'virtual_network_network_id': 7, 'virtual_network_properties': {'allow_transit': None, 'forwarding_mode': 'l3', 'max_flows': 0, 'mirror_destination': False, 'mtu': None, 'network_id': None, 'rpf': None, 'vxlan_network_identifier': None}}, 'oper': 'CREATE', 'request-id': 'req-59aee991-1a94-429a-b425-b319a40967e8', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:34:56.693 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:56.694 7fbb3ccb62c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.694 7fbb3ccb62c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:34:56.642429", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:56.694 7fbb3ccb62c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:56.714 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'] not found in DB: Unknown id: routing_instance default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network 09/01/2025 01:34:56.719 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:34:56.692301', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.692301', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11196397370808665147, 'uuid_mslong': 4939850386392500102}}, 'parent_type': 'security-group', 'parent_uuid': 'e0fa3e04-8145-47f6-9b69-718272132295', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'uuid': '448ddfc1-0480-4786-9b61-9816732b043b'}, 'oper': 'CREATE', 'request-id': 'req-226d9218-d6e4-44e4-8425-5c01a5bf858b', 'type': 'access_control_list', 'uuid': '448ddfc1-0480-4786-9b61-9816732b043b'} 09/01/2025 01:34:56.720 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = post body = {"routing-instance": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "parent_type": "virtual-network", "routing_instance_is_default": true, "routing_instance_has_pnf": false, "routing_instance_fabric_snat": true, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-pod-network"}} domain = default-domain >> 09/01/2025 01:34:56.760 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'obj_dict': {'display_name': 'k8s-default-pod-network', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'id_perms': {'created': '2025-09-01T01:34:56.741593', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.741593', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9857280386579676206, 'uuid_mslong': 6892182549909753541}}, 'parent_type': 'virtual-network', 'parent_uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': True, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'}, 'oper': 'CREATE', 'request-id': 'req-59aee991-1a94-429a-b425-b319a40967e8', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:56.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:56.780 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network 09/01/2025 01:34:56.783 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "parent_type": "virtual-network", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-pod-network"}} domain = default-domain >> 09/01/2025 01:34:56.868 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-pod-network'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-pod-network'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-pod-network'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'deny'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-pod-network'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000002'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'k8s-default-pod-network', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'id_perms': {'created': '2025-09-01T01:34:56.808303', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.808303', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12634401716669227668, 'uuid_mslong': 9014006186852894039}}, 'parent_type': 'virtual-network', 'parent_uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '7d182ede-628a-4557-af56-696bdfa7a294'}, 'oper': 'CREATE', 'request-id': 'req-1c8d35d4-fa79-4b1e-be60-502b66b7db78', 'type': 'access_control_list', 'uuid': '7d182ede-628a-4557-af56-696bdfa7a294'} 09/01/2025 01:34:56.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:34:56.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:34:56.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_refs in resource virtual-network: 'network_policy_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:56.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:34:56.875 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:34:56.881 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-service-ipam'] not found in DB: Unknown id: network_ipam default-domain:k8s-default:k8s-service-ipam 09/01/2025 01:34:56.884 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_ipam identifier_name = default-domain:k8s-default:k8s-service-ipam url = http://10.0.0.50:8082/network-ipams operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-ipam": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-service-ipam"], "parent_type": "project", "ipam_subnets": {"subnets": [{"subnet": {"ip_prefix": "10.96.0.0", "ip_prefix_len": 12}, "default_gateway": null, "dns_server_address": null, "subnet_uuid": null, "enable_dhcp": true, "dns_nameservers": [], "allocation_pools": [], "addr_from_start": null, "dhcp_option_list": null, "host_routes": null, "subnet_name": null, "alloc_unit": 1, "created": null, "last_modified": null, "subscriber_tag": null, "vlan_tag": null, "dhcp_relay_server": []}]}, "ipam_subnet_method": "flat-subnet", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-service-ipam"}} domain = default-domain >> 09/01/2025 01:34:56.886 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000004'] not found in DB: Unknown id: route_target target:64512:8000004 09/01/2025 01:34:56.886 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000004 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000004"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000004"}} domain = default-domain >> 09/01/2025 01:34:56.899 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000004'], 'obj_dict': {'display_name': 'target:64512:8000004', 'fq_name': ['target:64512:8000004'], 'id_perms': {'created': '2025-09-01T01:34:56.891155', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.891155', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13143918234955476838, 'uuid_mslong': 13547113033425112061}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bc010387-395e-4bfd-b668-93fec74cef66'}, 'oper': 'CREATE', 'request-id': 'req-32a5bfe9-d020-47b3-8268-e0dcf80f619e', 'type': 'route_target', 'uuid': 'bc010387-395e-4bfd-b668-93fec74cef66'} 09/01/2025 01:34:56.922 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-service-ipam'], 'obj_dict': {'display_name': 'k8s-service-ipam', 'fq_name': ['default-domain', 'k8s-default', 'k8s-service-ipam'], 'id_perms': {'created': '2025-09-01T01:34:56.901828', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:56.901828', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10632512379082133835, 'uuid_mslong': 17731803175260668569}}, 'ipam_subnet_method': 'flat-subnet', 'ipam_subnets': {'subnets': [{'addr_from_start': None, 'alloc_unit': 1, 'allocation_pools': [], 'created': None, 'default_gateway': '10.111.255.254', 'dhcp_option_list': None, 'dhcp_relay_server': [], 'dns_nameservers': [], 'dns_server_address': '10.111.255.253', 'enable_dhcp': True, 'host_routes': None, 'last_modified': None, 'subnet': {'ip_prefix': '10.96.0.0', 'ip_prefix_len': 12}, 'subnet_name': None, 'subnet_uuid': None, 'subscriber_tag': None, 'vlan_tag': None}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'f614050a-708e-4299-938e-45ace122514b'}, 'oper': 'CREATE', 'request-id': 'req-c80b4929-fcc0-4eb7-96d7-1f51333b6e1a', 'type': 'network_ipam', 'uuid': 'f614050a-708e-4299-938e-45ace122514b'} 09/01/2025 01:34:56.945 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e", "ref-type": "route-target", "ref-uuid": "bc010387-395e-4bfd-b668-93fec74cef66", "ref-fq-name": ["target:64512:8000004"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:56.955 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-e7ae3a1d-9b1b-4bd4-be39-41c1c5a4cdc5', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:56.958 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 7d182ede-628a-4557-af56-696bdfa7a294 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/access-control-list/7d182ede-628a-4557-af56-696bdfa7a294 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "7d182ede-628a-4557-af56-696bdfa7a294", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": -6805826841428061256}} domain = default-domain >> 09/01/2025 01:34:56.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-service-network'] not found in DB: Unknown id: virtual_network default-domain:k8s-default:k8s-default-service-network 09/01/2025 01:34:57.010 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-networks operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-service-network"], "parent_type": "project", "virtual_network_properties": {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null}, "fabric_snat": false, "address_allocation_mode": "flat-subnet-only", "network_ipam_refs": [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-service-network"}} domain = default-domain >> 09/01/2025 01:34:57.013 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-fec8e9ff-0f7e-4854-abee-9b44fc1073dc', 'type': 'access_control_list', 'uuid': '7d182ede-628a-4557-af56-696bdfa7a294'} 09/01/2025 01:34:57.098 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'obj_dict': {'address_allocation_mode': 'flat-subnet-only', 'display_name': 'k8s-default-service-network', 'fabric_snat': False, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'id_perms': {'created': '2025-09-01T01:34:57.045929', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.045929', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11820567914732569193, 'uuid_mslong': 12016753917161719144}}, 'is_shared': False, 'network_ipam_refs': [{'attr': {'host_routes': None, 'ipam_subnets': [{'subnet_uuid': 'cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400'}]}, 'to': ['default-domain', 'k8s-default', 'k8s-service-ipam'], 'uuid': 'f614050a-708e-4299-938e-45ace122514b'}], 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269', 'virtual_network_network_id': 8, 'virtual_network_properties': {'allow_transit': None, 'forwarding_mode': 'l3', 'max_flows': 0, 'mirror_destination': False, 'mtu': None, 'network_id': None, 'rpf': None, 'vxlan_network_identifier': None}}, 'oper': 'CREATE', 'request-id': 'req-232d5b26-5459-4cd8-acd5-13056167b54e', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:34:57.099 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVNTable:ContrailConfigTrace] map 09/01/2025 01:34:57.099 7fbb41e2d220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.099 7fbb41e2d220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:34:57.045929", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.099 7fbb41e2d220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'] not found in DB: Unknown id: routing_instance default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network 09/01/2025 01:34:57.116 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = post body = {"routing-instance": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-service-network", "k8s-default-service-network"], "parent_type": "virtual-network", "routing_instance_is_default": true, "routing_instance_has_pnf": false, "routing_instance_fabric_snat": false, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-service-network"}} domain = default-domain >> 09/01/2025 01:34:57.136 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'obj_dict': {'display_name': 'k8s-default-service-network', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'id_perms': {'created': '2025-09-01T01:34:57.129557', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.129557', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10058874526549187742, 'uuid_mslong': 11572430850722186631}}, 'parent_type': 'virtual-network', 'parent_uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'routing_instance_fabric_snat': False, 'routing_instance_has_pnf': False, 'routing_instance_is_default': True, 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'}, 'oper': 'CREATE', 'request-id': 'req-232d5b26-5459-4cd8-acd5-13056167b54e', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:34:57.155 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_refs in resource virtual-network: 'network_policy_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:34:57.156 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:34:57.165 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['target:64512:8000005'] not found in DB: Unknown id: route_target target:64512:8000005 09/01/2025 01:34:57.165 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = route_target identifier_name = target:64512:8000005 url = http://10.0.0.50:8082/route-targets operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"route-target": {"uuid": null, "fq_name": ["target:64512:8000005"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "target:64512:8000005"}} domain = default-domain >> 09/01/2025 01:34:57.176 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['target:64512:8000005'], 'obj_dict': {'display_name': 'target:64512:8000005', 'fq_name': ['target:64512:8000005'], 'id_perms': {'created': '2025-09-01T01:34:57.170046', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.170046', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13218141680492412823, 'uuid_mslong': 12160149632363022162}}, 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a8c187a9-d498-4752-b770-45d36088ff97'}, 'oper': 'CREATE', 'request-id': 'req-c62be96c-759a-46f4-a171-07ccd4bfb462', 'type': 'route_target', 'uuid': 'a8c187a9-d498-4752-b770-45d36088ff97'} 09/01/2025 01:34:57.181 7fbb3ce50220 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:34:57.182 7fbb3ce50220 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:34:57.183 7fbb3ce50220 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.186 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "a099888b-9d11-4587-8b98-4d1673309c9e", "ref-type": "route-target", "ref-uuid": "a8c187a9-d498-4752-b770-45d36088ff97", "ref-fq-name": ["target:64512:8000005"], "operation": "ADD", "attr": {"import_export": null}} domain = default-domain >> 09/01/2025 01:34:57.195 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-565d0cba-c30e-48eb-aea3-a2632f8f2e7a', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:57.213 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-ip-fabric-np'] not found in DB: Unknown id: network_policy default-domain:k8s-default:k8s-default-ip-fabric-np 09/01/2025 01:34:57.215 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_policy identifier_name = default-domain:k8s-default:k8s-default-ip-fabric-np url = http://10.0.0.50:8082/network-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-policy": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "parent_type": "project", "network_policy_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": null, "direction": "<>", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": null, "network_policy": "default-domain:k8s-default:k8s-default-ip-fabric-np", "subnet_list": []}], "src_ports": [{"start_port": -1, "end_port": -1}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": -1, "end_port": -1}], "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "ethertype": null, "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-ip-fabric-np"}} domain = default-domain >> 09/01/2025 01:34:57.231 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-ip-fabric-np'], 'obj_dict': {'display_name': 'k8s-default-ip-fabric-np', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-ip-fabric-np'], 'id_perms': {'created': '2025-09-01T01:34:57.224637', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.224637', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9820500363855709707, 'uuid_mslong': 6871729834595271054}}, 'network_policy_entries': {'policy_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'application': [], 'created': None, 'description': None, 'direction': '<>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}], 'dst_ports': [{'end_port': -1, 'start_port': -1}], 'ethertype': None, 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '8f3cc2a6-3f21-40bd-9cf8-07332875b75a', 'src_addresses': [{'network_policy': 'default-domain:k8s-default:k8s-default-ip-fabric-np', 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': -1, 'start_port': -1}]}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '5f5d49e0-6781-418e-8849-6d0cb836c60b'}, 'oper': 'CREATE', 'request-id': 'req-e1542e63-440c-4bfd-bd1d-9ed52a8104eb', 'type': 'network_policy', 'uuid': '5f5d49e0-6781-418e-8849-6d0cb836c60b'} 09/01/2025 01:34:57.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-service-np'] not found in DB: Unknown id: network_policy default-domain:k8s-default:k8s-default-service-np 09/01/2025 01:34:57.279 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_policy identifier_name = default-domain:k8s-default:k8s-default-service-np url = http://10.0.0.50:8082/network-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-policy": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-service-np"], "parent_type": "project", "network_policy_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": null, "direction": "<>", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": null, "network_policy": "default-domain:k8s-default:k8s-default-service-np", "subnet_list": []}], "src_ports": [{"start_port": -1, "end_port": -1}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": -1, "end_port": -1}], "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "ethertype": null, "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-service-np"}} domain = default-domain >> 09/01/2025 01:34:57.304 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-np'], 'obj_dict': {'display_name': 'k8s-default-service-np', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-np'], 'id_perms': {'created': '2025-09-01T01:34:57.294843', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.294843', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12937301099142653850, 'uuid_mslong': 11970738815292819317}}, 'network_policy_entries': {'policy_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'application': [], 'created': None, 'description': None, 'direction': '<>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-service-network'}], 'dst_ports': [{'end_port': -1, 'start_port': -1}], 'ethertype': None, 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '2cc90ab6-4fd1-4d50-8836-f841cc9e0762', 'src_addresses': [{'network_policy': 'default-domain:k8s-default:k8s-default-service-np', 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': -1, 'start_port': -1}]}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'a6209b87-6082-4775-b38a-86ce0eab279a'}, 'oper': 'CREATE', 'request-id': 'req-734b6617-5e04-4ff6-99ab-f28409d88552', 'type': 'network_policy', 'uuid': 'a6209b87-6082-4775-b38a-86ce0eab279a'} 09/01/2025 01:34:57.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-pod-service-np'] not found in DB: Unknown id: network_policy default-domain:k8s-default:k8s-default-pod-service-np 09/01/2025 01:34:57.328 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = network_policy identifier_name = default-domain:k8s-default:k8s-default-pod-service-np url = http://10.0.0.50:8082/network-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-policy": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "parent_type": "project", "network_policy_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": null, "direction": "<>", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": -1, "end_port": -1}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": -1, "end_port": -1}], "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "ethertype": null, "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-pod-service-np"}} domain = default-domain >> 09/01/2025 01:34:57.345 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-service-np'], 'obj_dict': {'display_name': 'k8s-default-pod-service-np', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-service-np'], 'id_perms': {'created': '2025-09-01T01:34:57.336300', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.336300', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9434725081410064776, 'uuid_mslong': 2241212454514476764}}, 'network_policy_entries': {'policy_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'application': [], 'created': None, 'description': None, 'direction': '<>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-service-network'}], 'dst_ports': [{'end_port': -1, 'start_port': -1}], 'ethertype': None, 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '629589cc-223a-407e-a160-df2b0b524fff', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-pod-network'}], 'src_ports': [{'end_port': -1, 'start_port': -1}]}]}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '1f1a62da-5984-46dc-82ee-e07966ea5188'}, 'oper': 'CREATE', 'request-id': 'req-104165ae-0cd3-4d5b-9ae2-ca203fee1e4e', 'type': 'network_policy', 'uuid': '1f1a62da-5984-46dc-82ee-e07966ea5188'} 09/01/2025 01:34:57.394 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e07e6e04-4e93-4043-ab83-32b8f8854bbe object_type = virtual_network identifier_name = default-domain:default-project:ip-fabric url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe", "ref-type": "network-policy", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.413 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'default-project', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-21093259-5a40-4df3-bbb7-8edf25d90339', 'type': 'virtual_network', 'uuid': 'e07e6e04-4e93-4043-ab83-32b8f8854bbe'} 09/01/2025 01:34:57.425 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:34:57.426 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:34:57.427 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:34:57.429 7fbb3ce50540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.429 7fbb3ce50540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:default-project:ip-fabric elements = { fabric_snat : falseflood_unknown_unicast : falsefq_name : ["default-domain", "default-project", "ip-fabric"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16176487876338597955, "uuid_lslong": 12358777572459629502}, "enable": true, "created": "2025-09-01T01:34:46.683844", "last_modified": "2025-09-01T01:34:57.402804", "description": null, "user_visible": true, "creator": null}igmp_enable : falseis_provider_network : trueis_shared : falselayer2_control_word : falsemac_aging_time : 300mac_ip_learning_enable : falsemac_learning_enabled : falsenetwork_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "55ddbd9e-8af0-4bf7-97b7-49fc60f8bda4"pbb_etree_enable : falsepbb_evpn_enable : falseperms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : trueuuid : "e07e6e04-4e93-4043-ab83-32b8f8854bbe"virtual_network_category : "tenant"virtual_network_network_id : 2 } deleted = False >> 09/01/2025 01:34:57.429 7fbb3ce50540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.430 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 23054292-38bc-4b6f-b5fa-0f8021a7362f object_type = routing_instance identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:57.430 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://10.0.0.50:8082/access-control-list/a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": -7613866323721909924}} domain = default-domain >> 09/01/2025 01:34:57.447 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-b1969f07-11f0-4f5a-acfd-26d748871cb4', 'type': 'access_control_list', 'uuid': 'a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce'} 09/01/2025 01:34:57.451 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-21093259-5a40-4df3-bbb7-8edf25d90339', 'type': 'routing_instance', 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'} 09/01/2025 01:34:57.465 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e07e6e04-4e93-4043-ab83-32b8f8854bbe object_type = virtual_network identifier_name = default-domain:default-project:ip-fabric url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"} domain = default-domain >> 09/01/2025 01:34:57.471 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-type": "network-policy", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.479 7fbb3ce502c0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: config_database_node default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:34:57.485 7fbb3ce502c0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = config_database_node identifier_name = default-global-system-config:cn-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/config-database-nodes operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_config_database_node.py remote_ip = 10.0.0.50 body = {"config-database-node": {"uuid": null, "fq_name": ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "config_database_node_ip_address": "10.0.0.50", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "cn-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:34:57.490 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-780f1e29-426f-4c8a-95bd-66a39eb92d4d', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:34:57.500 7fbb3ce50b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.500 7fbb3ce50b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:34:57.481567", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.500 7fbb3ce50b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.504 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://10.0.0.50:8082/access-control-list/a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": 7690074450295442098}} domain = default-domain >> 09/01/2025 01:34:57.505 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:57.519 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-1b6a50f4-d11d-4775-8b07-24c851eab479', 'type': 'access_control_list', 'uuid': 'a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce'} 09/01/2025 01:34:57.524 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'config_database_node_ip_address': '10.0.0.50', 'display_name': 'cn-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'cn-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:34:57.511328', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.511328', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10288526971317814616, 'uuid_mslong': 9619987740220280603}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '85810fe1-7f71-4f1b-8ec8-30c4fb6c8558'}, 'oper': 'CREATE', 'request-id': 'req-bc40be73-7cda-43db-bf51-5d88a64adeb5', 'type': 'config_database_node', 'uuid': '85810fe1-7f71-4f1b-8ec8-30c4fb6c8558'} 09/01/2025 01:34:57.524 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectConfigDatabaseInfo:ContrailConfigTrace] map 09/01/2025 01:34:57.525 7fbb3ce50b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.525 7fbb3ce50b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 elements = { config_database_node_ip_address : "10.0.0.50"display_name : "cn-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "cn-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 9619987740220280603, "uuid_lslong": 10288526971317814616}, "enable": true, "created": "2025-09-01T01:34:57.511328", "last_modified": "2025-09-01T01:34:57.511328", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "85810fe1-7f71-4f1b-8ec8-30c4fb6c8558" } deleted = False >> 09/01/2025 01:34:57.525 7fbb3ce50b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.531 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 7d182ede-628a-4557-af56-696bdfa7a294 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/access-control-list/7d182ede-628a-4557-af56-696bdfa7a294 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "7d182ede-628a-4557-af56-696bdfa7a294", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": 5389381109424321708}} domain = default-domain >> 09/01/2025 01:34:57.544 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-780f1e29-426f-4c8a-95bd-66a39eb92d4d', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:57.615 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-type": "network-policy", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.617 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-63d9d2e6-ba1a-4d0d-8a27-1157f1443220', 'type': 'access_control_list', 'uuid': '7d182ede-628a-4557-af56-696bdfa7a294'} 09/01/2025 01:34:57.618 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e", "ref-type": "routing-instance", "ref-uuid": "23054292-38bc-4b6f-b5fa-0f8021a7362f", "ref-fq-name": null, "operation": "ADD", "attr": {"destination_instance": null}} domain = default-domain >> 09/01/2025 01:34:57.632 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-ed81ae90-4f21-4fc8-85dc-fb2d26a6f954', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:34:57.640 7fbb3ce50b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.641 7fbb3ce50b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:34:57.625401", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.641 7fbb3ce50b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.658 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:57.662 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-f01d4e77-4220-4d4d-b01b-775c7abb911e', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:57.663 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-f01d4e77-4220-4d4d-b01b-775c7abb911e', 'type': 'routing_instance', 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'} 09/01/2025 01:34:57.666 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 7d182ede-628a-4557-af56-696bdfa7a294 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://10.0.0.50:8082/access-control-list/7d182ede-628a-4557-af56-696bdfa7a294 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "7d182ede-628a-4557-af56-696bdfa7a294", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "629589cc-223a-407e-a160-df2b0b524fff", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": 6597944806087765751}} domain = default-domain >> 09/01/2025 01:34:57.695 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"} domain = default-domain >> 09/01/2025 01:34:57.696 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-ed81ae90-4f21-4fc8-85dc-fb2d26a6f954', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:57.708 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-37941629-f84d-418f-9a40-ad7b740bf2eb', 'type': 'access_control_list', 'uuid': '7d182ede-628a-4557-af56-696bdfa7a294'} 09/01/2025 01:34:57.709 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"} domain = default-domain >> 09/01/2025 01:34:57.759 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-type": "network-policy", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:57.773 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-7513ce5c-b4bf-4702-9bf3-443ac5b9bfb9', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:34:57.778 7fbb3ce50400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.784 7fbb3ce50400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:34:57.764756", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.785 7fbb3ce50400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.787 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:57.791 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce object_type = access_control_list identifier_name = default-domain:default-project:ip-fabric:ip-fabric url = http://10.0.0.50:8082/access-control-list/a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "deny", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000002", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": -2748703458065908071}} domain = default-domain >> 09/01/2025 01:34:57.804 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-7513ce5c-b4bf-4702-9bf3-443ac5b9bfb9', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:57.808 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE', 'request-id': 'req-93c437c6-6633-4b6e-a8ae-a1e511faedc6', 'type': 'access_control_list', 'uuid': 'a87aabc0-ac59-4e06-a71b-5e1b9d1a40ce'} 09/01/2025 01:34:57.816 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'] not found in DB: Unknown id: access_control_list default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network 09/01/2025 01:34:57.824 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "k8s-default-service-network", "k8s-default-service-network"], "parent_type": "virtual-network", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-default-service-network"}} domain = default-domain >> 09/01/2025 01:34:57.827 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-type": "network-policy", "ref-uuid": "a6209b87-6082-4775-b38a-86ce0eab279a", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-service-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.851 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-66ce2b60-91ef-454f-92d1-0a96443b837e', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:34:57.863 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "a099888b-9d11-4587-8b98-4d1673309c9e", "ref-type": "routing-instance", "ref-uuid": "23054292-38bc-4b6f-b5fa-0f8021a7362f", "ref-fq-name": null, "operation": "ADD", "attr": {"destination_instance": null}} domain = default-domain >> 09/01/2025 01:34:57.874 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.874 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:34:57.837833", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.874 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.882 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:57.889 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '<>', 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:default-project:ip-fabric'}, 'dst_port': {'end_port': -1, 'start_port': -1}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-service-network'}, 'src_port': {'end_port': -1, 'start_port': -1}}, 'rule_uuid': '8f3cc2a6-3f21-40bd-9cf8-07332875b75a'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-service-network'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'default-domain:k8s-default:k8s-default-service-network'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': None, 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': 'any'}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '00000000-0000-0000-0000-100000000001'}], 'dynamic': False}, 'display_name': 'k8s-default-service-network', 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'id_perms': {'created': '2025-09-01T01:34:57.838228', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:57.838228', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9688595357425536257, 'uuid_mslong': 962755083897160870}}, 'parent_type': 'virtual-network', 'parent_uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '0d5c64a5-dd64-44a6-8674-ce252bd90d01'}, 'oper': 'CREATE', 'request-id': 'req-0a977f31-731a-4fd1-a5df-e760cf5106de', 'type': 'access_control_list', 'uuid': '0d5c64a5-dd64-44a6-8674-ce252bd90d01'} 09/01/2025 01:34:57.908 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-1d8b72b9-3623-4b84-9199-10245016ba49', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:57.911 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'default-project', 'ip-fabric', 'ip-fabric'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-1d8b72b9-3623-4b84-9199-10245016ba49', 'type': 'routing_instance', 'uuid': '23054292-38bc-4b6f-b5fa-0f8021a7362f'} 09/01/2025 01:34:57.923 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-66ce2b60-91ef-454f-92d1-0a96443b837e', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:57.928 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-type": "network-policy", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:34:57.953 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-852aa536-24e0-4a8c-b109-95a6f9a39b58', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:34:57.965 7fbb3ce50b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.965 7fbb3ce50b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:34:57.943374", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:34:57.965 7fbb3ce50b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:57.970 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0d5c64a5-dd64-44a6-8674-ce252bd90d01 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://10.0.0.50:8082/access-control-list/0d5c64a5-dd64-44a6-8674-ce252bd90d01 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "0d5c64a5-dd64-44a6-8674-ce252bd90d01", "access_control_list_entries": {"dynamic": false, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-pod-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "629589cc-223a-407e-a160-df2b0b524fff", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:default-project:ip-fabric", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8f3cc2a6-3f21-40bd-9cf8-07332875b75a", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": -1, "end_port": -1}, "dst_address": {"subnet": null, "virtual_network": "default-domain:k8s-default:k8s-default-service-network", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": -1, "end_port": -1}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "2cc90ab6-4fd1-4d50-8836-f841cc9e0762", "direction": "<>"}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": "any", "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": null}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "00000000-0000-0000-0000-100000000001", "direction": null}]}, "access_control_list_hash": 6464964879923921046}} domain = default-domain >> 09/01/2025 01:34:57.986 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:34:58.007 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-6395e580-30d8-495c-ac31-7bebb8e20dad', 'type': 'access_control_list', 'uuid': '0d5c64a5-dd64-44a6-8674-ce252bd90d01'} 09/01/2025 01:34:58.014 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"type": "routing-instance", "uuid": "a099888b-9d11-4587-8b98-4d1673309c9e", "ref-type": "routing-instance", "ref-uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e", "ref-fq-name": null, "operation": "ADD", "attr": {"destination_instance": null}} domain = default-domain >> 09/01/2025 01:34:58.026 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-852aa536-24e0-4a8c-b109-95a6f9a39b58', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:58.041 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-5ea10595-094d-412f-bbae-a518ae5ea01c', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:34:58.043 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE-IMPLICIT', 'request-id': 'req-5ea10595-094d-412f-bbae-a518ae5ea01c', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:34:58.067 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"} domain = default-domain >> 09/01/2025 01:34:58.071 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"} domain = default-domain >> 09/01/2025 01:34:58.076 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"} domain = default-domain >> 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-network: 'network_ipam_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:34:58.153 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:34:58.154 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:34:58.188 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_refs in resource application-policy-set: 'firewall_policy_refs' 09/01/2025 01:34:58.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:58.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:58.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:34:58.243 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:34:58.248 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-denyall'] not found in DB: Unknown id: firewall_policy default-policy-management:k8s-denyall 09/01/2025 01:34:58.251 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_policy identifier_name = default-policy-management:k8s-denyall url = http://10.0.0.50:8082/firewall-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-policy": {"uuid": null, "fq_name": ["default-policy-management", "k8s-denyall"], "parent_type": "policy-management", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-denyall"}} domain = default-domain >> 09/01/2025 01:34:58.279 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-denyall'], 'obj_dict': {'display_name': 'k8s-denyall', 'fq_name': ['default-policy-management', 'k8s-denyall'], 'id_perms': {'created': '2025-09-01T01:34:58.269037', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:58.269037', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9281677744765653977, 'uuid_mslong': 16276300850529848719}}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e1e10963-5538-4d8f-80cf-24bb8067d3d9'}, 'oper': 'CREATE', 'request-id': 'req-013bb0d9-3a7e-4f51-9114-ec4159349c3c', 'type': 'firewall_policy', 'uuid': 'e1e10963-5538-4d8f-80cf-24bb8067d3d9'} 09/01/2025 01:34:58.281 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallPolicyTable:ContrailConfigTrace] map 09/01/2025 01:34:58.281 7fbb3ce50720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.281 7fbb3ce50720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-denyall elements = { display_name : "k8s-denyall"fq_name : ["default-policy-management", "k8s-denyall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16276300850529848719, "uuid_lslong": 9281677744765653977}, "enable": true, "created": "2025-09-01T01:34:58.269037", "last_modified": "2025-09-01T01:34:58.269037", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e1e10963-5538-4d8f-80cf-24bb8067d3d9" } deleted = False >> 09/01/2025 01:34:58.281 7fbb3ce50720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.328 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e1e10963-5538-4d8f-80cf-24bb8067d3d9 object_type = firewall_policy identifier_name = default-policy-management:k8s-denyall url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "VncSecurityPolicy"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": null}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": null}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "denyall"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": null}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "spec", "value": "null"}, "position": "spec"}, {"field": "annotations", "operation": "set", "value": {"key": "tail", "value": "True"}, "position": "tail"}]} domain = default-domain >> 09/01/2025 01:34:58.339 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-denyall'], 'oper': 'UPDATE', 'request-id': 'req-51c82e2c-a312-42d1-b532-2aa30f0232b1', 'type': 'firewall_policy', 'uuid': 'e1e10963-5538-4d8f-80cf-24bb8067d3d9'} 09/01/2025 01:34:58.341 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.341 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.342 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.342 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.343 7fbb3ce50400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.343 7fbb3ce50400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-denyall elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "denyall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}, {"key": "tail", "value": "True"}]}display_name : "k8s-denyall"fq_name : ["default-policy-management", "k8s-denyall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16276300850529848719, "uuid_lslong": 9281677744765653977}, "enable": true, "created": "2025-09-01T01:34:58.269037", "last_modified": "2025-09-01T01:34:58.332533", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "e1e10963-5538-4d8f-80cf-24bb8067d3d9" } deleted = False >> 09/01/2025 01:34:58.343 7fbb3ce50400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.346 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_refs in resource application-policy-set: 'firewall_policy_refs' 09/01/2025 01:34:58.346 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:58.346 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:58.346 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:58.395 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.419 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.419 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.419 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.426 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "ref-fq-name": ["default-policy-management", "k8s-denyall"], "operation": "ADD", "attr": {"sequence": "00001.0"}} domain = default-domain >> 09/01/2025 01:34:58.453 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-bccd7199-869c-49f9-bc42-a24d93d7a01e', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:58.460 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.460 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00001.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:58.444417", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:58.460 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.461 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:58.461 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:58.461 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:34:58.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:34:58.511 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:34:58.514 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-allowall'] not found in DB: Unknown id: firewall_policy default-policy-management:k8s-allowall 09/01/2025 01:34:58.516 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/firewall-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-policy": {"uuid": null, "fq_name": ["default-policy-management", "k8s-allowall"], "parent_type": "policy-management", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-allowall"}} domain = default-domain >> 09/01/2025 01:34:58.620 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'obj_dict': {'display_name': 'k8s-allowall', 'fq_name': ['default-policy-management', 'k8s-allowall'], 'id_perms': {'created': '2025-09-01T01:34:58.612519', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:58.612519', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11791485297466611927, 'uuid_mslong': 12591360059846116675}}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'}, 'oper': 'CREATE', 'request-id': 'req-eacc1f31-9300-4f08-9293-9539003f9be2', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:34:58.621 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallPolicyTable:ContrailConfigTrace] map 09/01/2025 01:34:58.621 7fbb3ce50680 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.622 7fbb3ce50680 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { display_name : "k8s-allowall"fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:34:58.612519", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:34:58.622 7fbb3ce50680 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.667 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "VncSecurityPolicy"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": null}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": null}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "allowall"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": null}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "spec", "value": "null"}, "position": "spec"}, {"field": "annotations", "operation": "set", "value": {"key": "after_tail", "value": "True"}, "position": "after_tail"}]} domain = default-domain >> 09/01/2025 01:34:58.694 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-1f44f737-0863-4947-b5c3-f31c406646c1', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:34:58.698 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.699 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.699 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.699 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.701 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.702 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:34:58.684883", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:34:58.702 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:58.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:58.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:58.755 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.755 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.755 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.755 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:58.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.772 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:58.772 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:58.772 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:58.772 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:58.827 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "ref-fq-name": ["default-policy-management", "k8s-denyall"], "operation": "DELETE", "attr": null} domain = default-domain >> 09/01/2025 01:34:58.839 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-c6b60776-7032-491d-a74d-53aa89fe5a38', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:58.848 7fbb3ce50720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.848 7fbb3ce50720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:58.834351", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:58.848 7fbb3ce50720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.854 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "ref-fq-name": ["default-policy-management", "k8s-denyall"], "operation": "ADD", "attr": {"sequence": "00002.0"}} domain = default-domain >> 09/01/2025 01:34:58.877 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-7b527ba4-12f9-4141-957d-e71ac80e0977', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:58.885 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.885 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00002.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:58.868862", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:58.885 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.889 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-fq-name": ["default-policy-management", "k8s-allowall"], "operation": "ADD", "attr": {"sequence": "00003.0"}} domain = default-domain >> 09/01/2025 01:34:58.914 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-7c6f2258-bcda-4d16-9dba-264fa9d38c4e', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:58.916 7fbb3ce50720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.916 7fbb3ce50720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00003.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}, {"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00002.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:58.910132", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:58.916 7fbb3ce50720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:58.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:58.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:58.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:34:58.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:34:59.019 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Ingress'] not found in DB: Unknown id: firewall_policy default-policy-management:k8s-Ingress 09/01/2025 01:34:59.022 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_policy identifier_name = default-policy-management:k8s-Ingress url = http://10.0.0.50:8082/firewall-policys operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-policy": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Ingress"], "parent_type": "policy-management", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Ingress"}} domain = default-domain >> 09/01/2025 01:34:59.060 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Ingress'], 'obj_dict': {'display_name': 'k8s-Ingress', 'fq_name': ['default-policy-management', 'k8s-Ingress'], 'id_perms': {'created': '2025-09-01T01:34:59.040105', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:59.040105', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10188735552209959056, 'uuid_mslong': 7715603300617307372}}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '6b135452-a920-44ec-8d65-a900a3de3890'}, 'oper': 'CREATE', 'request-id': 'req-c0edc76c-a3f8-449f-b64c-b3aad4442d98', 'type': 'firewall_policy', 'uuid': '6b135452-a920-44ec-8d65-a900a3de3890'} 09/01/2025 01:34:59.060 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallPolicyTable:ContrailConfigTrace] map 09/01/2025 01:34:59.061 7fbb3ce50720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.061 7fbb3ce50720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Ingress elements = { display_name : "k8s-Ingress"fq_name : ["default-policy-management", "k8s-Ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7715603300617307372, "uuid_lslong": 10188735552209959056}, "enable": true, "created": "2025-09-01T01:34:59.040105", "last_modified": "2025-09-01T01:34:59.040105", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "6b135452-a920-44ec-8d65-a900a3de3890" } deleted = False >> 09/01/2025 01:34:59.061 7fbb3ce50720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.072 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6b135452-a920-44ec-8d65-a900a3de3890 object_type = firewall_policy identifier_name = default-policy-management:k8s-Ingress url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "6b135452-a920-44ec-8d65-a900a3de3890", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "VncSecurityPolicy"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": null}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": null}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "Ingress"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": null}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "spec", "value": "null"}, "position": "spec"}]} domain = default-domain >> 09/01/2025 01:34:59.083 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Ingress'], 'oper': 'UPDATE', 'request-id': 'req-c7a474ad-d2ab-43c0-a434-7c283897e862', 'type': 'firewall_policy', 'uuid': '6b135452-a920-44ec-8d65-a900a3de3890'} 09/01/2025 01:34:59.087 7fbb3ce50400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.088 7fbb3ce50400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Ingress elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "Ingress"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-Ingress"fq_name : ["default-policy-management", "k8s-Ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7715603300617307372, "uuid_lslong": 10188735552209959056}, "enable": true, "created": "2025-09-01T01:34:59.040105", "last_modified": "2025-09-01T01:34:59.076039", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "6b135452-a920-44ec-8d65-a900a3de3890" } deleted = False >> 09/01/2025 01:34:59.088 7fbb3ce50400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.135 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:59.135 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:59.135 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:59.139 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.143 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.143 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.143 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.144 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.196 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.205 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:34:59.205 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:34:59.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:34:59.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:34:59.255 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "ref-fq-name": ["default-policy-management", "k8s-denyall"], "operation": "DELETE", "attr": null} domain = default-domain >> 09/01/2025 01:34:59.277 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-5d95b5ce-064e-4fb8-a834-2b173c32b6cb', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.317 7fbb3ce50720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.318 7fbb3ce50720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00003.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.263953", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.318 7fbb3ce50720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.320 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-fq-name": ["default-policy-management", "k8s-allowall"], "operation": "DELETE", "attr": null} domain = default-domain >> 09/01/2025 01:34:59.344 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-206b120c-3131-4ae7-8935-b0755f77ee9f', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.346 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.346 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.325886", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.346 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.351 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-fq-name": ["default-policy-management", "k8s-allowall"], "operation": "ADD", "attr": {"sequence": "00005.0"}} domain = default-domain >> 09/01/2025 01:34:59.366 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-3444dad5-67b5-4b4c-8fcc-0d82e1938fe5', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.370 7fbb3ce50400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.370 7fbb3ce50400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00005.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.360278", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.370 7fbb3ce50400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.415 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9", "ref-fq-name": ["default-policy-management", "k8s-denyall"], "operation": "ADD", "attr": {"sequence": "00004.0"}} domain = default-domain >> 09/01/2025 01:34:59.437 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-80f3d144-97b9-4630-ab6d-f6c12edc818a', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.492 7fbb3ce50860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.493 7fbb3ce50860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00005.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}, {"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00004.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.430670", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.493 7fbb3ce50860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.500 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "application-policy-set", "uuid": "1a9b0278-098b-4ce5-aee7-f5f921fa367d", "ref-type": "firewall-policy", "ref-uuid": "6b135452-a920-44ec-8d65-a900a3de3890", "ref-fq-name": ["default-policy-management", "k8s-Ingress"], "operation": "ADD", "attr": {"sequence": "00002.0"}} domain = default-domain >> 09/01/2025 01:34:59.528 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-95997700-60ae-48d9-9394-8ef1de58aca0', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.532 7fbb3ce50400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.532 7fbb3ce50400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-Ingress"], "attr": {"sequence": "00002.0"}, "uuid": "6b135452-a920-44ec-8d65-a900a3de3890"}, {"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00005.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}, {"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00004.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.522503", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.533 7fbb3ce50400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.546 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:34:59.546 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource application-policy-set: 'tag_refs' 09/01/2025 01:34:59.546 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:34:59.595 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['application=k8s'] not found in DB: Unknown id: tag application=k8s 09/01/2025 01:34:59.595 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = application=k8s url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["application=k8s"], "tag_type_name": "application", "tag_value": "k8s", "name": "application=k8s", "display_name": "application=k8s", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:34:59.748 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['application=k8s'], 'obj_dict': {'display_name': 'application=k8s', 'fq_name': ['application=k8s'], 'id_perms': {'created': '2025-09-01T01:34:59.735253', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:59.735253', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11942659685020229637, 'uuid_mslong': 2992592746691382189}}, 'name': 'application=k8s', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x00010000', 'tag_type_name': 'application', 'tag_type_refs': [{'to': ['application'], 'uuid': 'd092075d-9887-4533-9bdd-86b77d6c3c87'}], 'tag_value': 'k8s', 'uuid': '2987d33f-67f3-4bad-a5bc-d9b567026005'}, 'oper': 'CREATE', 'request-id': 'req-74ba4f4e-fe97-4ec0-bfd1-f8040e55371b', 'type': 'tag', 'uuid': '2987d33f-67f3-4bad-a5bc-d9b567026005'} 09/01/2025 01:34:59.748 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:34:59.749 7fbb3ce50540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.749 7fbb3ce50540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = application=k8s elements = { display_name : "application=k8s"fq_name : ["application=k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 2992592746691382189, "uuid_lslong": 11942659685020229637}, "enable": true, "created": "2025-09-01T01:34:59.735253", "last_modified": "2025-09-01T01:34:59.735253", "description": null, "user_visible": true, "creator": null}name : "application=k8s"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x00010000"tag_type_name : "application"tag_type_refs : [{"uuid": "d092075d-9887-4533-9bdd-86b77d6c3c87", "to": ["application"]}]tag_value : "k8s"uuid : "2987d33f-67f3-4bad-a5bc-d9b567026005" } deleted = False >> 09/01/2025 01:34:59.749 7fbb3ce50540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:34:59.762 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:34:59.763 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:34:59.764 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:34:59.765 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:34:59.766 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:34:59.767 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:34:59.768 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:34:59.769 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:34:59.770 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:34:59.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:34:59.821 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1a9b0278-098b-4ce5-aee7-f5f921fa367d object_type = application_policy_set identifier_name = default-policy-management:k8s url = http://127.0.0.1/application-policy-sets operation = http_put domain = default-domain >> 09/01/2025 01:34:59.835 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s'], 'oper': 'UPDATE', 'request-id': 'req-4d9bbb9e-f9da-425a-a88c-ad9a8d083f2b', 'type': 'application_policy_set', 'uuid': '1a9b0278-098b-4ce5-aee7-f5f921fa367d'} 09/01/2025 01:34:59.841 7fbb3ce50360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.841 7fbb3ce50360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s elements = { display_name : "k8s"firewall_policy_refs : [{"to": ["default-policy-management", "k8s-Ingress"], "attr": {"sequence": "00002.0"}, "uuid": "6b135452-a920-44ec-8d65-a900a3de3890"}, {"to": ["default-policy-management", "k8s-allowall"], "attr": {"sequence": "00005.0"}, "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7"}, {"to": ["default-policy-management", "k8s-denyall"], "attr": {"sequence": "00004.0"}, "uuid": "e1e10963-5538-4d8f-80cf-24bb8067d3d9"}]fq_name : ["default-policy-management", "k8s"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1917128780955733221, "uuid_lslong": 12603312532585330301}, "enable": true, "created": "2025-09-01T01:34:55.834723", "last_modified": "2025-09-01T01:34:59.829452", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "1a9b0278-098b-4ce5-aee7-f5f921fa367d" } deleted = False >> 09/01/2025 01:34:59.841 7fbb3ce50360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.897 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s-app=kube-dns'] not found in DB: Unknown id: tag k8s-app=kube-dns 09/01/2025 01:34:59.898 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = k8s-app=kube-dns url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["k8s-app=kube-dns"], "tag_type_name": "k8s-app", "tag_value": "kube-dns", "name": "k8s-app=kube-dns", "display_name": "k8s-app=kube-dns", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:34:59.919 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s-app'] not found in DB: Unknown id: tag_type k8s-app 09/01/2025 01:34:59.919 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag_type identifier_name = k8s-app url = http://127.0.0.1/tag-types operation = post body = {"tag-type": {"uuid": null, "fq_name": ["k8s-app"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": null, "created": null, "last_modified": null, "description": null, "user_visible": false, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-app"}} domain = default-domain >> 09/01/2025 01:34:59.943 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s-app'], 'obj_dict': {'display_name': 'k8s-app', 'fq_name': ['k8s-app'], 'id_perms': {'created': '2025-09-01T01:34:59.932934', 'creator': None, 'description': None, 'enable': None, 'last_modified': '2025-09-01T01:34:59.932934', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 13013349798405736565, 'uuid_mslong': 6378882733062310404}}, 'name': 'k8s-app', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x00ff', 'uuid': '5886580c-880d-4a04-b498-b4b1a7e00875'}, 'oper': 'CREATE', 'request-id': 'req-dd1a0a6b-707c-4067-8461-f611ba652626', 'type': 'tag_type', 'uuid': '5886580c-880d-4a04-b498-b4b1a7e00875'} 09/01/2025 01:34:59.968 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s-app=kube-dns'], 'obj_dict': {'display_name': 'k8s-app=kube-dns', 'fq_name': ['k8s-app=kube-dns'], 'id_perms': {'created': '2025-09-01T01:34:59.959558', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:34:59.959558', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11471135153113387167, 'uuid_mslong': 7836802123086514403}}, 'name': 'k8s-app=kube-dns', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x00ff0000', 'tag_type_name': 'k8s-app', 'tag_type_refs': [{'to': ['k8s-app'], 'uuid': '5886580c-880d-4a04-b498-b4b1a7e00875'}], 'tag_value': 'kube-dns', 'uuid': '6cc1ea02-8199-48e3-9f31-a8a523d1d09f'}, 'oper': 'CREATE', 'request-id': 'req-dd1a0a6b-707c-4067-8461-f611ba652626', 'type': 'tag', 'uuid': '6cc1ea02-8199-48e3-9f31-a8a523d1d09f'} 09/01/2025 01:34:59.968 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:34:59.969 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.969 7fbb3ce504a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = k8s-app=kube-dns elements = { display_name : "k8s-app=kube-dns"fq_name : ["k8s-app=kube-dns"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7836802123086514403, "uuid_lslong": 11471135153113387167}, "enable": true, "created": "2025-09-01T01:34:59.959558", "last_modified": "2025-09-01T01:34:59.959558", "description": null, "user_visible": true, "creator": null}name : "k8s-app=kube-dns"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x00ff0000"tag_type_name : "k8s-app"tag_type_refs : [{"uuid": "5886580c-880d-4a04-b498-b4b1a7e00875", "to": ["k8s-app"]}]tag_value : "kube-dns"uuid : "6cc1ea02-8199-48e3-9f31-a8a523d1d09f" } deleted = False >> 09/01/2025 01:34:59.969 7fbb3ce504a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:34:59.984 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:34:59.985 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:34:59.986 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:34:59.987 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:34:59.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:34:59.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:34:59.990 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:34:59.991 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:34:59.992 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:34:59.993 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:34:59.993 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:34:59.993 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:34:59.993 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:34:59.993 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:00.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['pod-template-hash=6f867cd986'] not found in DB: Unknown id: tag pod-template-hash=6f867cd986 09/01/2025 01:35:00.039 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = pod-template-hash=6f867cd986 url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["pod-template-hash=6f867cd986"], "tag_type_name": "pod-template-hash", "tag_value": "6f867cd986", "name": "pod-template-hash=6f867cd986", "display_name": "pod-template-hash=6f867cd986", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:00.047 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['pod-template-hash'] not found in DB: Unknown id: tag_type pod-template-hash 09/01/2025 01:35:00.047 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag_type identifier_name = pod-template-hash url = http://127.0.0.1/tag-types operation = post body = {"tag-type": {"uuid": null, "fq_name": ["pod-template-hash"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": null, "created": null, "last_modified": null, "description": null, "user_visible": false, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "pod-template-hash"}} domain = default-domain >> 09/01/2025 01:35:00.058 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['pod-template-hash'], 'obj_dict': {'display_name': 'pod-template-hash', 'fq_name': ['pod-template-hash'], 'id_perms': {'created': '2025-09-01T01:35:00.054172', 'creator': None, 'description': None, 'enable': None, 'last_modified': '2025-09-01T01:35:00.054172', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11925374521531939020, 'uuid_mslong': 7275911611303085316}}, 'name': 'pod-template-hash', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0100', 'uuid': '64f93b0f-a0dc-4504-a57f-70f1a967b8cc'}, 'oper': 'CREATE', 'request-id': 'req-ecafa1a4-ff9e-46b3-91a7-b2758a53c848', 'type': 'tag_type', 'uuid': '64f93b0f-a0dc-4504-a57f-70f1a967b8cc'} 09/01/2025 01:35:00.095 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['pod-template-hash=6f867cd986'], 'obj_dict': {'display_name': 'pod-template-hash=6f867cd986', 'fq_name': ['pod-template-hash=6f867cd986'], 'id_perms': {'created': '2025-09-01T01:35:00.081848', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:00.081848', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13788021986930285393, 'uuid_mslong': 6374837122979416772}}, 'name': 'pod-template-hash=6f867cd986', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01000000', 'tag_type_name': 'pod-template-hash', 'tag_type_refs': [{'to': ['pod-template-hash'], 'uuid': '64f93b0f-a0dc-4504-a57f-70f1a967b8cc'}], 'tag_value': '6f867cd986', 'uuid': '5877f896-8829-4ac4-bf58-e4f1ee415b51'}, 'oper': 'CREATE', 'request-id': 'req-ecafa1a4-ff9e-46b3-91a7-b2758a53c848', 'type': 'tag', 'uuid': '5877f896-8829-4ac4-bf58-e4f1ee415b51'} 09/01/2025 01:35:00.095 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:00.096 7fbb3ce50c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.096 7fbb3ce50c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = pod-template-hash=6f867cd986 elements = { display_name : "pod-template-hash=6f867cd986"fq_name : ["pod-template-hash=6f867cd986"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6374837122979416772, "uuid_lslong": 13788021986930285393}, "enable": true, "created": "2025-09-01T01:35:00.081848", "last_modified": "2025-09-01T01:35:00.081848", "description": null, "user_visible": true, "creator": null}name : "pod-template-hash=6f867cd986"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01000000"tag_type_name : "pod-template-hash"tag_type_refs : [{"uuid": "64f93b0f-a0dc-4504-a57f-70f1a967b8cc", "to": ["pod-template-hash"]}]tag_value : "6f867cd986"uuid : "5877f896-8829-4ac4-bf58-e4f1ee415b51" } deleted = False >> 09/01/2025 01:35:00.096 7fbb3ce50c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:00.111 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:00.112 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:00.113 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:00.114 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:00.115 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:00.116 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:00.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:00.166 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace=kube-system'] not found in DB: Unknown id: tag namespace=kube-system 09/01/2025 01:35:00.167 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = namespace=kube-system url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["namespace=kube-system"], "tag_type_name": "namespace", "tag_value": "kube-system", "name": "namespace=kube-system", "display_name": "namespace=kube-system", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:00.168 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 09/01/2025 01:35:00.168 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 09/01/2025 01:35:00.168 7fbb3ceb7860 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 09/01/2025 01:35:00.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace'] not found in DB: Unknown id: tag_type namespace 09/01/2025 01:35:00.176 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag_type identifier_name = namespace url = http://127.0.0.1/tag-types operation = post body = {"tag-type": {"uuid": null, "fq_name": ["namespace"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": null, "created": null, "last_modified": null, "description": null, "user_visible": false, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "namespace"}} domain = default-domain >> 09/01/2025 01:35:00.188 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace'], 'obj_dict': {'display_name': 'namespace', 'fq_name': ['namespace'], 'id_perms': {'created': '2025-09-01T01:35:00.184338', 'creator': None, 'description': None, 'enable': None, 'last_modified': '2025-09-01T01:35:00.184338', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 12142692889164708494, 'uuid_mslong': 9821876152153425061}}, 'name': 'namespace', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0101', 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}, 'oper': 'CREATE', 'request-id': 'req-c8d73cb7-47c0-47d5-a681-44a5a299ad3c', 'type': 'tag_type', 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'} 09/01/2025 01:35:00.227 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace=kube-system'], 'obj_dict': {'display_name': 'namespace=kube-system', 'fq_name': ['namespace=kube-system'], 'id_perms': {'created': '2025-09-01T01:35:00.212283', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:00.212283', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9786979529561711537, 'uuid_mslong': 3566125101318032622}}, 'name': 'namespace=kube-system', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01010000', 'tag_type_name': 'namespace', 'tag_type_refs': [{'to': ['namespace'], 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}], 'tag_value': 'kube-system', 'uuid': '317d6be2-b0aa-4cee-87d2-56074b8ba3b1'}, 'oper': 'CREATE', 'request-id': 'req-c8d73cb7-47c0-47d5-a681-44a5a299ad3c', 'type': 'tag', 'uuid': '317d6be2-b0aa-4cee-87d2-56074b8ba3b1'} 09/01/2025 01:35:00.227 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:00.228 7fbb3ce50c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.229 7fbb3ce50c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = namespace=kube-system elements = { display_name : "namespace=kube-system"fq_name : ["namespace=kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 3566125101318032622, "uuid_lslong": 9786979529561711537}, "enable": true, "created": "2025-09-01T01:35:00.212283", "last_modified": "2025-09-01T01:35:00.212283", "description": null, "user_visible": true, "creator": null}name : "namespace=kube-system"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01010000"tag_type_name : "namespace"tag_type_refs : [{"uuid": "884e5052-64fc-44a5-a883-82d91352fa8e", "to": ["namespace"]}]tag_value : "kube-system"uuid : "317d6be2-b0aa-4cee-87d2-56074b8ba3b1" } deleted = False >> 09/01/2025 01:35:00.229 7fbb3ce50c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.231 7fbb3ceb7860 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 87a687b0-9781-4994-aa87-dbc3f38997c0 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = http://10.0.0.50:8082/global-vrouter-config/87a687b0-9781-4994-aa87-dbc3f38997c0 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_linklocal.py remote_ip = 10.0.0.50 body = {"global-vrouter-config": {"uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "linklocal_services": {"linklocal_service_entry": [{"linklocal_service_name": "metadata", "linklocal_service_ip": "169.254.169.254", "linklocal_service_port": 80, "ip_fabric_DNS_service_name": "", "ip_fabric_service_port": 8775, "ip_fabric_service_ip": ["127.0.0.1"]}]}}} domain = default-domain >> 09/01/2025 01:35:00.245 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-2a369105-28c6-49cb-be5b-da7741b80510', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:35:00.255 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:00.255 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:00.255 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.256 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:00.257 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:00.258 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:00.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:00.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:00.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:00.263 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:00.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:00.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:00.266 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:00.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:00.268 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:00.269 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:00.336 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:00.337 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:00.338 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:00.339 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:00.392 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:00.400 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:00.431 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-6d90e0f3-59f2-487e-b779-b1873bf72868', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:00.437 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:00.438 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:00.439 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:00.440 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:00.441 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:00.443 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:00.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:00.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:00.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:00.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:00.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:00.450 7fbb3ceb7f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.450 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:00.420851", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:00.450 7fbb3ceb7f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.455 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s__kube-system__coredns-6f867cd986-wlkqx'] not found in DB: Unknown id: virtual_machine k8s__kube-system__coredns-6f867cd986-wlkqx 09/01/2025 01:35:00.457 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8 not found in DB: Unknown id: a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8 09/01/2025 01:35:00.457 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8 object_type = virtual_machine identifier_name = k8s__kube-system__coredns-6f867cd986-wlkqx url = http://10.0.0.50:8082/virtual-machines operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-machine": {"uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8", "fq_name": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "server_type": "container", "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s__kube-system__coredns-6f867cd986-wlkqx", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:00.458 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-conf-elements-inconsistency'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-conf-elements-inconsistency 09/01/2025 01:35:00.461 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-conf-elements-inconsistency url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-conf-elements-inconsistency"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements", "operation": "size==", "operand2": {"json_value": "1"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "ContrailConfig has a list of elements - it should be a dict. Controllers have inconsistency in their data.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-conf-elements-inconsistency"}} domain = default-domain >> 09/01/2025 01:35:00.466 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s__kube-system__coredns-6f867cd986-wlkqx'], 'obj_dict': {'display_name': 'k8s__kube-system__coredns-6f867cd986-wlkqx', 'fq_name': ['k8s__kube-system__coredns-6f867cd986-wlkqx'], 'id_perms': {'created': '2025-09-01T01:35:00.459243', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:00.459243', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12979318260264720552, 'uuid_mslong': 11863649037683543068}}, 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'server_type': 'container', 'uuid': 'a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8'}, 'oper': 'CREATE', 'request-id': 'req-c7d91bb6-6d43-4eb5-a9e6-750581fac2d9', 'type': 'virtual_machine', 'uuid': 'a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8'} 09/01/2025 01:35:00.466 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVMTable:ContrailConfigTrace] map 09/01/2025 01:35:00.467 7fbb3ce50e00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.467 7fbb3ce50e00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = k8s__kube-system__coredns-6f867cd986-wlkqx elements = { display_name : "k8s__kube-system__coredns-6f867cd986-wlkqx"fq_name : ["k8s__kube-system__coredns-6f867cd986-wlkqx"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11863649037683543068, "uuid_lslong": 12979318260264720552}, "enable": true, "created": "2025-09-01T01:35:00.459243", "last_modified": "2025-09-01T01:35:00.459243", "description": null, "user_visible": true, "creator": null}perms2 : {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}server_type : "container"uuid : "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8" } deleted = False >> 09/01/2025 01:35:00.467 7fbb3ce50e00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.483 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8 object_type = virtual_machine identifier_name = k8s__kube-system__coredns-6f867cd986-wlkqx url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Pod"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-system"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-system"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "labels", "value": "{\"k8s-app\": \"kube-dns\", \"pod-template-hash\": \"6f867cd986\", \"namespace\": \"kube-system\"}"}, "position": "labels"}]} domain = default-domain >> 09/01/2025 01:35:00.497 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements', 'operand2': {'json_value': '1'}, 'operation': 'size=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-elements-inconsistency', 'fq_name': ['default-global-system-config', 'system-defined-conf-elements-inconsistency'], 'id_perms': {'created': '2025-09-01T01:35:00.482458', 'creator': None, 'description': 'ContrailConfig has a list of ' 'elements - it should be a dict. ' 'Controllers have inconsistency in ' 'their data.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.482458', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11541890835256865210, 'uuid_mslong': 5907038753012272917}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '51fa0472-f3c3-4f15-a02d-089027d271ba', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-7e298936-cd5c-4a6e-b167-019153a81134', 'type': 'alarm', 'uuid': '51fa0472-f3c3-4f15-a02d-089027d271ba'} 09/01/2025 01:35:00.501 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-address-mismatch-compute'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-address-mismatch-compute 09/01/2025 01:35:00.502 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s__kube-system__coredns-6f867cd986-wlkqx'], 'oper': 'UPDATE', 'request-id': 'req-785a52a2-faa6-404a-a42b-4d1b7282c546', 'type': 'virtual_machine', 'uuid': 'a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8'} 09/01/2025 01:35:00.502 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'] not found in DB: Unknown id: virtual_machine_interface default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d 09/01/2025 01:35:00.508 7fbb3ce50ea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.508 7fbb3ce50ea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = k8s__kube-system__coredns-6f867cd986-wlkqx elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}, {"key": "kind", "value": "Pod"}, {"key": "labels", "value": "{\"k8s-app\": \"kube-dns\", \"pod-template-hash\": \"6f867cd986\", \"namespace\": \"kube-system\"}"}, {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, {"key": "namespace", "value": "kube-system"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-system"}]}display_name : "k8s__kube-system__coredns-6f867cd986-wlkqx"fq_name : ["k8s__kube-system__coredns-6f867cd986-wlkqx"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11863649037683543068, "uuid_lslong": 12979318260264720552}, "enable": true, "created": "2025-09-01T01:35:00.459243", "last_modified": "2025-09-01T01:35:00.484780", "description": null, "user_visible": true, "creator": null}perms2 : {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}server_type : "container"uuid : "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8" } deleted = False >> 09/01/2025 01:35:00.508 7fbb3ce50ea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.509 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-address-mismatch-compute url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-address-mismatch-compute"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_ip_address", "operation": "!=", "operand2": {"uve_attribute": "VrouterAgent.control_ip"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Compute Node IP Address mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-address-mismatch-compute"}} domain = default-domain >> 09/01/2025 01:35:00.513 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID e02048c4-86d3-11f0-93e1-fa163e599e0d not found in DB: Unknown id: e02048c4-86d3-11f0-93e1-fa163e599e0d 09/01/2025 01:35:00.513 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e02048c4-86d3-11f0-93e1-fa163e599e0d object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d url = http://10.0.0.50:8082/virtual-machine-interfaces operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-machine-interface": {"uuid": "e02048c4-86d3-11f0-93e1-fa163e599e0d", "fq_name": ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"], "parent_type": "project", "port_security_enabled": true, "display_name": "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d", "security_group_refs": [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"]}], "virtual_machine_refs": [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}], "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:00.527 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_ip_address', 'operand2': {'uve_attribute': 'VrouterAgent.control_ip'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-compute', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-compute'], 'id_perms': {'created': '2025-09-01T01:35:00.518275', 'creator': None, 'description': 'Compute Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.518275', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13335252558768919498, 'uuid_mslong': 17592579791041153399}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f425661c-47b9-4977-b910-558c8b8a53ca', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-9ac55cb3-b339-4003-889d-4986b65d2c3c', 'type': 'alarm', 'uuid': 'f425661c-47b9-4977-b910-558c8b8a53ca'} 09/01/2025 01:35:00.530 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-address-mismatch-control'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-address-mismatch-control 09/01/2025 01:35:00.533 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-address-mismatch-control url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-address-mismatch-control"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.bgp_router_parameters.address", "operation": "not in", "operand2": {"uve_attribute": "BgpRouterState.bgp_router_ip_list"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Control Node IP Address mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-address-mismatch-control"}} domain = default-domain >> 09/01/2025 01:35:00.604 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'obj_dict': {'display_name': 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d', 'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'id_perms': {'created': '2025-09-01T01:35:00.522540', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:00.522540', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10656073166777589261, 'uuid_mslong': 16149988272663368176}}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'security_group_refs': [{'to': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg']}], 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d', 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:e0:20:48:c4:86']}, 'virtual_machine_refs': [{'to': ['k8s__kube-system__coredns-6f867cd986-wlkqx'], 'uuid': 'a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8'}], 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'}]}, 'oper': 'CREATE', 'request-id': 'req-854ab66d-b57e-4e35-8305-1d0f374869a0', 'type': 'virtual_machine_interface', 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.605 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVMITable:ContrailConfigTrace] map 09/01/2025 01:35:00.609 7fbb3ced6040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.609 7fbb3ced6040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d elements = { display_name : "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"fq_name : ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16149988272663368176, "uuid_lslong": 10656073166777589261}, "enable": true, "created": "2025-09-01T01:35:00.522540", "last_modified": "2025-09-01T01:35:00.522540", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truesecurity_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"]}]uuid : "e02048c4-86d3-11f0-93e1-fa163e599e0d"virtual_machine_interface_mac_addresses : {"mac_address": ["02:e0:20:48:c4:86"]}virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}] } deleted = False >> 09/01/2025 01:35:00.609 7fbb3ced6040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.612 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e02048c4-86d3-11f0-93e1-fa163e599e0d object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:00.619 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.bgp_router_parameters.address', 'operand2': {'uve_attribute': 'BgpRouterState.bgp_router_ip_list'}, 'operation': 'not ' 'in'}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-address-mismatch-control', 'fq_name': ['default-global-system-config', 'system-defined-address-mismatch-control'], 'id_perms': {'created': '2025-09-01T01:35:00.591651', 'creator': None, 'description': 'Control Node IP Address mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.591651', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13765896304333387515, 'uuid_mslong': 12941592107138171089}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'b399c574-35f8-40d1-bf0a-49c102a776fb', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-6f69ecda-97a5-4292-87aa-f9505f715fd1', 'type': 'alarm', 'uuid': 'b399c574-35f8-40d1-bf0a-49c102a776fb'} 09/01/2025 01:35:00.632 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'oper': 'UPDATE', 'request-id': 'req-854ab66d-b57e-4e35-8305-1d0f374869a0', 'type': 'virtual_machine_interface', 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.644 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.644 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d elements = { display_name : "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"fq_name : ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16149988272663368176, "uuid_lslong": 10656073166777589261}, "enable": true, "created": "2025-09-01T01:35:00.522540", "last_modified": "2025-09-01T01:35:00.627178", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e"}]security_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "attr": null, "uuid": "e0fa3e04-8145-47f6-9b69-718272132295"}]uuid : "e02048c4-86d3-11f0-93e1-fa163e599e0d"virtual_machine_interface_mac_addresses : {"mac_address": ["02:e0:20:48:c4:86"]}virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "attr": null, "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "attr": null, "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}] } deleted = False >> 09/01/2025 01:35:00.644 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.666 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-bgp-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-bgp-connectivity 09/01/2025 01:35:00.668 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-bgp-connectivity url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-bgp-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "BgpRouterState.num_up_bgp_peer", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "BgpRouterState.num_up_bgp_peer", "operation": "!=", "operand2": {"uve_attribute": "BgpRouterState.num_bgp_peer"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "BGP peer mismatch. Not enough BGP peers are up.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-bgp-connectivity"}} domain = default-domain >> 09/01/2025 01:35:00.674 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e02048c4-86d3-11f0-93e1-fa163e599e0d object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "e02048c4-86d3-11f0-93e1-fa163e599e0d", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Pod"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-system"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-system"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "index", "value": "0/1"}, "position": "index"}, {"field": "annotations", "operation": "set", "value": {"key": "network", "value": "default"}, "position": "network"}]} domain = default-domain >> 09/01/2025 01:35:00.682 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'oper': 'UPDATE', 'request-id': 'req-341bddab-ceb6-434c-b328-52a1c8c3f011', 'type': 'virtual_machine_interface', 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.695 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_bgp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_bgp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-bgp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-bgp-connectivity'], 'id_perms': {'created': '2025-09-01T01:35:00.676695', 'creator': None, 'description': 'BGP peer mismatch. Not enough BGP ' 'peers are up.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.676695', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12737114632070279124, 'uuid_mslong': 3323528339732513001}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2e1f8b6a-8ab3-40e9-b0c3-524613faafd4', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-403323e3-fe22-4014-99d0-048dab956108', 'type': 'alarm', 'uuid': '2e1f8b6a-8ab3-40e9-b0c3-524613faafd4'} 09/01/2025 01:35:00.696 7fbb3ce50ea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.696 7fbb3ce50ea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "index", "value": "0/1"}, {"key": "kind", "value": "Pod"}, {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, {"key": "namespace", "value": "kube-system"}, {"key": "network", "value": "default"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-system"}]}display_name : "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"fq_name : ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16149988272663368176, "uuid_lslong": 10656073166777589261}, "enable": true, "created": "2025-09-01T01:35:00.522540", "last_modified": "2025-09-01T01:35:00.676268", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e"}]security_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "attr": null, "uuid": "e0fa3e04-8145-47f6-9b69-718272132295"}]uuid : "e02048c4-86d3-11f0-93e1-fa163e599e0d"virtual_machine_interface_mac_addresses : {"mac_address": ["02:e0:20:48:c4:86"]}virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "attr": null, "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "attr": null, "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}] } deleted = False >> 09/01/2025 01:35:00.696 7fbb3ce50ea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.701 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-conf-incorrect'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-conf-incorrect 09/01/2025 01:35:00.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:00.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:00.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:00.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:00.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:00.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:00.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:00.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:00.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:00.710 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-conf-incorrect url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-conf-incorrect"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "ContrailConfig missing or incorrect. Configuration pushed to Ifmap as ContrailConfig is missing/incorrect.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-conf-incorrect"}} domain = default-domain >> 09/01/2025 01:35:00.747 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-conf-incorrect', 'fq_name': ['default-global-system-config', 'system-defined-conf-incorrect'], 'id_perms': {'created': '2025-09-01T01:35:00.737556', 'creator': None, 'description': 'ContrailConfig missing or ' 'incorrect. Configuration pushed to ' 'Ifmap as ContrailConfig is ' 'missing/incorrect.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.737556', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10437081795927609670, 'uuid_mslong': 1742884343849304489}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '182ff813-97ba-41a9-90d7-f69869f61146', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-ce389d40-228e-4de7-877a-338c3d9f547c', 'type': 'alarm', 'uuid': '182ff813-97ba-41a9-90d7-f69869f61146'} 09/01/2025 01:35:00.748 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-disk-usage-high'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-disk-usage-high 09/01/2025 01:35:00.755 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d'] not found in DB: Unknown id: instance_ip coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d 09/01/2025 01:35:00.757 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-disk-usage-high url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-disk-usage-high"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.disk_usage_info.*.percentage_partition_space_used", "operation": "range", "operand2": {"json_value": "[70, 90]"}, "variables": ["NodeStatus.disk_usage_info.__key"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Disk usage crosses high threshold limit.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-disk-usage-high"}} domain = default-domain >> 09/01/2025 01:35:00.760 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID e03cee52-86d3-11f0-93e1-fa163e599e0d not found in DB: Unknown id: e03cee52-86d3-11f0-93e1-fa163e599e0d 09/01/2025 01:35:00.760 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e03cee52-86d3-11f0-93e1-fa163e599e0d object_type = instance_ip identifier_name = coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d url = http://10.0.0.50:8082/instance-ips operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"instance-ip": {"uuid": "e03cee52-86d3-11f0-93e1-fa163e599e0d", "fq_name": ["coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d"], "subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9", "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d", "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}], "virtual_machine_interface_refs": [{"to": ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"], "uuid": "e02048c4-86d3-11f0-93e1-fa163e599e0d"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:00.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:00.783 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '[70, ' '90]'}, 'operation': 'range', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-disk-usage-high', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-high'], 'id_perms': {'created': '2025-09-01T01:35:00.770077', 'creator': None, 'description': 'Disk usage crosses high threshold ' 'limit.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.770077', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10283240903023742618, 'uuid_mslong': 17457322688003264184}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f244de7d-ebcd-4eb8-8eb5-691e4c7f4e9a', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a4f088e2-148a-4e77-8192-29645683a9a4', 'type': 'alarm', 'uuid': 'f244de7d-ebcd-4eb8-8eb5-691e4c7f4e9a'} 09/01/2025 01:35:00.808 7fbb3ceb7400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: AddrMgmt: alloc IIP 10.47.255.252 for vn=['default-domain', 'k8s-default', 'k8s-default-pod-network'], tenant=default-project, askip=None 09/01/2025 01:35:00.819 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-disk-usage-critical'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-disk-usage-critical 09/01/2025 01:35:00.825 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-disk-usage-critical url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-disk-usage-critical"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.disk_usage_info.*.percentage_partition_space_used", "operation": ">", "operand2": {"json_value": "90"}, "variables": ["NodeStatus.disk_usage_info.__key"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Disk usage crosses critical threshold limit.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-disk-usage-critical"}} domain = default-domain >> 09/01/2025 01:35:00.849 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d'], 'obj_dict': {'display_name': 'coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d', 'fq_name': ['coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d'], 'id_perms': {'created': '2025-09-01T01:35:00.818422', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:00.818422', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10656073166777589261, 'uuid_mslong': 16158051601315205616}}, 'instance_ip_address': '10.47.255.252', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'subnet_uuid': 'eada5f2d-448e-4621-9fea-812925eb46e9', 'uuid': 'e03cee52-86d3-11f0-93e1-fa163e599e0d', 'virtual_machine_interface_refs': [{'to': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'}], 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'}]}, 'oper': 'CREATE', 'request-id': 'req-c80d919e-d72c-4d2d-b5cf-ce84e19a49b9', 'type': 'instance_ip', 'uuid': 'e03cee52-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.858 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e03cee52-86d3-11f0-93e1-fa163e599e0d object_type = instance_ip identifier_name = coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "e03cee52-86d3-11f0-93e1-fa163e599e0d", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Pod"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-system"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-system"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:00.873 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.disk_usage_info.*.percentage_partition_space_used', 'operand2': {'json_value': '90'}, 'operation': '>', 'variables': ['NodeStatus.disk_usage_info.__key']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-disk-usage-critical', 'fq_name': ['default-global-system-config', 'system-defined-disk-usage-critical'], 'id_perms': {'created': '2025-09-01T01:35:00.861644', 'creator': None, 'description': 'Disk usage crosses critical ' 'threshold limit.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.861644', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12788391283575139026, 'uuid_mslong': 5764728618455484405}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '50006e22-b991-47f5-b179-7e1dda32aad2', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-5e38e384-5e93-4efc-b6a4-b22ad6729fbf', 'type': 'alarm', 'uuid': '50006e22-b991-47f5-b179-7e1dda32aad2'} 09/01/2025 01:35:00.875 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-node-status'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-node-status 09/01/2025 01:35:00.883 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['coredns-6f867cd986-wlkqx__e03cee52-86d3-11f0-93e1-fa163e599e0d'], 'oper': 'UPDATE', 'request-id': 'req-cad95a26-b166-494b-93d6-2b3f5d2e7f85', 'type': 'instance_ip', 'uuid': 'e03cee52-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.887 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-node-status url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-node-status"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Node Failure. NodeStatus UVE not present.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-node-status"}} domain = default-domain >> 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:00.888 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:00.889 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:00.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:00.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:00.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:00.902 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-node-status', 'fq_name': ['default-global-system-config', 'system-defined-node-status'], 'id_perms': {'created': '2025-09-01T01:35:00.897537', 'creator': None, 'description': 'Node Failure. NodeStatus UVE not ' 'present.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.897537', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9800546595679803193, 'uuid_mslong': 16768712725561229374}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'e8b66f61-23a6-403e-8802-893408289739', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-68e47281-94a2-4cf6-9860-a1a9cffe35b4', 'type': 'alarm', 'uuid': 'e8b66f61-23a6-403e-8802-893408289739'} 09/01/2025 01:35:00.908 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-partial-sysinfo'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-partial-sysinfo 09/01/2025 01:35:00.912 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-partial-sysinfo url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-partial-sysinfo"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.build_info", "operation": "==", "operand2": {"json_value": "null"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "System Info Incomplete.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-partial-sysinfo"}} domain = default-domain >> 09/01/2025 01:35:00.925 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.build_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-partial-sysinfo', 'fq_name': ['default-global-system-config', 'system-defined-partial-sysinfo'], 'id_perms': {'created': '2025-09-01T01:35:00.919411', 'creator': None, 'description': 'System Info Incomplete.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.919411', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9841848269247336815, 'uuid_mslong': 13773325918658710460}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'bf24aef2-e555-4bbc-8895-44db45c7d16f', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-468f6b69-7fe0-47a7-881d-24e979235447', 'type': 'alarm', 'uuid': 'bf24aef2-e555-4bbc-8895-44db45c7d16f'} 09/01/2025 01:35:00.951 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e02048c4-86d3-11f0-93e1-fa163e599e0d object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d url = http://127.0.0.1/virtual-machine-interfaces operation = http_put domain = default-domain >> 09/01/2025 01:35:00.968 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'oper': 'UPDATE', 'request-id': 'req-5cb9c65d-a555-4752-be33-f345bd871e71', 'type': 'virtual_machine_interface', 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:00.971 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-process-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-process-connectivity 09/01/2025 01:35:00.974 7fbb3ce50f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.974 7fbb3ce50f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "index", "value": "0/1"}, {"key": "kind", "value": "Pod"}, {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, {"key": "namespace", "value": "kube-system"}, {"key": "network", "value": "default"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-system"}]}display_name : "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"fq_name : ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16149988272663368176, "uuid_lslong": 10656073166777589261}, "enable": true, "created": "2025-09-01T01:35:00.522540", "last_modified": "2025-09-01T01:35:00.957084", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e"}]security_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "attr": null, "uuid": "e0fa3e04-8145-47f6-9b69-718272132295"}]tag_refs : [{"to": ["namespace=kube-system"], "attr": null, "uuid": "317d6be2-b0aa-4cee-87d2-56074b8ba3b1"}, {"to": ["pod-template-hash=6f867cd986"], "attr": null, "uuid": "5877f896-8829-4ac4-bf58-e4f1ee415b51"}, {"to": ["k8s-app=kube-dns"], "attr": null, "uuid": "6cc1ea02-8199-48e3-9f31-a8a523d1d09f"}]uuid : "e02048c4-86d3-11f0-93e1-fa163e599e0d"virtual_machine_interface_mac_addresses : {"mac_address": ["02:e0:20:48:c4:86"]}virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "attr": null, "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "attr": null, "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}] } deleted = False >> 09/01/2025 01:35:00.974 7fbb3ce50f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:00.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:00.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:00.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:00.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:00.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:00.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:00.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:00.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:00.979 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:00.980 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:00.982 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-process-connectivity url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-process-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.process_status", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "NodeStatus.process_status.state", "operation": "!=", "operand2": {"json_value": "\"Functional\""}, "variables": ["NodeStatus.process_status.module_id", "NodeStatus.process_status.instance_id"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Process(es) reporting as non-functional.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-process-connectivity"}} domain = default-domain >> 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:00.988 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:00.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:00.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:00.989 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:01.000 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_status', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_status.state', 'operand2': {'json_value': '"Functional"'}, 'operation': '!=', 'variables': ['NodeStatus.process_status.module_id', 'NodeStatus.process_status.instance_id']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-process-connectivity'], 'id_perms': {'created': '2025-09-01T01:35:00.993525', 'creator': None, 'description': 'Process(es) reporting as ' 'non-functional.', 'enable': True, 'last_modified': '2025-09-01T01:35:00.993525', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13655082713157644973, 'uuid_mslong': 7422783952441593023}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '670306ad-59c0-4cbf-bd80-99614056b2ad', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-553602d1-d201-4b11-8569-96d2a7801aee', 'type': 'alarm', 'uuid': '670306ad-59c0-4cbf-bd80-99614056b2ad'} 09/01/2025 01:35:01.005 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-process-status'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-process-status 09/01/2025 01:35:01.014 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-process-status url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-process-status"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.process_info", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "NodeStatus.process_info.process_state", "operation": "!=", "operand2": {"json_value": "\"PROCESS_STATE_RUNNING\""}, "variables": ["NodeStatus.process_info.process_name"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Process Failure.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-process-status"}} domain = default-domain >> 09/01/2025 01:35:01.030 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.process_info', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'NodeStatus.process_info.process_state', 'operand2': {'json_value': '"PROCESS_STATE_RUNNING"'}, 'operation': '!=', 'variables': ['NodeStatus.process_info.process_name']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-process-status', 'fq_name': ['default-global-system-config', 'system-defined-process-status'], 'id_perms': {'created': '2025-09-01T01:35:01.022450', 'creator': None, 'description': 'Process Failure.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.022450', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10351739637149028015, 'uuid_mslong': 894098479270019581}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '0c6879d4-6b1b-41fd-8fa8-c45aa7994eaf', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-d87fd2d0-ee20-4f9c-aa86-fa97c52766aa', 'type': 'alarm', 'uuid': '0c6879d4-6b1b-41fd-8fa8-c45aa7994eaf'} 09/01/2025 01:35:01.039 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-prouter-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-prouter-connectivity 09/01/2025 01:35:01.052 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-prouter-connectivity url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-prouter-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["prouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.connected_agent_list", "operation": "size!=", "operand2": {"json_value": "1"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Prouter connectivity to controlling tor agent does not exist we look for non-empty value for connected_agent_list", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-prouter-connectivity"}} domain = default-domain >> 09/01/2025 01:35:01.063 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name=contrail'] not found in DB: Unknown id: tag kubernetes.io__slash__metadata.name=contrail 09/01/2025 01:35:01.063 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = kubernetes.io__slash__metadata.name=contrail url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name=contrail"], "tag_type_name": "kubernetes.io__slash__metadata.name", "tag_value": "contrail", "name": "kubernetes.io__slash__metadata.name=contrail", "display_name": "kubernetes.io__slash__metadata.name=contrail", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:01.071 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.connected_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-connectivity'], 'id_perms': {'created': '2025-09-01T01:35:01.060992', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tor agent does not exist we look ' 'for non-empty value for ' 'connected_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:35:01.060992', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10335860996523724715, 'uuid_mslong': 16042134643040011702}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'dea11c76-8563-45b6-8f70-5ad0899c8fab', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-a70d16ac-9e75-4a7b-abc2-b8ad36bbe09e', 'type': 'alarm', 'uuid': 'dea11c76-8563-45b6-8f70-5ad0899c8fab'} 09/01/2025 01:35:01.080 7fbb3ce50f40 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:35:01.081 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name'] not found in DB: Unknown id: tag_type kubernetes.io__slash__metadata.name 09/01/2025 01:35:01.081 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag_type identifier_name = kubernetes.io__slash__metadata.name url = http://127.0.0.1/tag-types operation = post body = {"tag-type": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": null, "created": null, "last_modified": null, "description": null, "user_visible": false, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kubernetes.io__slash__metadata.name"}} domain = default-domain >> 09/01/2025 01:35:01.083 7fbb3ce50ae0 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:35:01.091 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name', 'fq_name': ['kubernetes.io__slash__metadata.name'], 'id_perms': {'created': '2025-09-01T01:35:01.087385', 'creator': None, 'description': None, 'enable': None, 'last_modified': '2025-09-01T01:35:01.087385', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 13828626881080767734, 'uuid_mslong': 14178771421114287520}}, 'name': 'kubernetes.io__slash__metadata.name', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0102', 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}, 'oper': 'CREATE', 'request-id': 'req-e4e030dc-7426-4998-a3af-f25f1827bff9', 'type': 'tag_type', 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'} 09/01/2025 01:35:01.110 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-prouter-tsn-connectivity 09/01/2025 01:35:01.112 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-prouter-tsn-connectivity url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-prouter-tsn-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["prouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.tsn_agent_list", "operation": "size!=", "operand2": {"json_value": "1"}}, {"operand1": "ProuterData.gateway_mode", "operation": "!=", "operand2": {"json_value": "\"SERVER\""}}]}, {"and_list": [{"operand1": "ContrailConfig.elements.virtual_router_refs", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "ProuterData.tsn_agent_list", "operation": "size!=", "operand2": {"json_value": "0"}}, {"operand1": "ProuterData.gateway_mode", "operation": "==", "operand2": {"json_value": "\"SERVER\""}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Prouter connectivity to controlling tsn agent does not exist we look for non-empty value for tsn_agent_list", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-prouter-tsn-connectivity"}} domain = default-domain >> 09/01/2025 01:35:01.127 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name=contrail'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name=contrail', 'fq_name': ['kubernetes.io__slash__metadata.name=contrail'], 'id_perms': {'created': '2025-09-01T01:35:01.111225', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:01.111225', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12675199517905652019, 'uuid_mslong': 11957672635128762339}}, 'name': 'kubernetes.io__slash__metadata.name=contrail', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01020000', 'tag_type_name': 'kubernetes.io__slash__metadata.name', 'tag_type_refs': [{'to': ['kubernetes.io__slash__metadata.name'], 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}], 'tag_value': 'contrail', 'uuid': 'a5f22fe8-3d42-4fe3-afe7-5ace31f80933'}, 'oper': 'CREATE', 'request-id': 'req-e4e030dc-7426-4998-a3af-f25f1827bff9', 'type': 'tag', 'uuid': 'a5f22fe8-3d42-4fe3-afe7-5ace31f80933'} 09/01/2025 01:35:01.127 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:01.128 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.128 7fbb3ce504a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = kubernetes.io__slash__metadata.name=contrail elements = { display_name : "kubernetes.io__slash__metadata.name=contrail"fq_name : ["kubernetes.io__slash__metadata.name=contrail"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11957672635128762339, "uuid_lslong": 12675199517905652019}, "enable": true, "created": "2025-09-01T01:35:01.111225", "last_modified": "2025-09-01T01:35:01.111225", "description": null, "user_visible": true, "creator": null}name : "kubernetes.io__slash__metadata.name=contrail"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01020000"tag_type_name : "kubernetes.io__slash__metadata.name"tag_type_refs : [{"uuid": "c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6", "to": ["kubernetes.io__slash__metadata.name"]}]tag_value : "contrail"uuid : "a5f22fe8-3d42-4fe3-afe7-5ace31f80933" } deleted = False >> 09/01/2025 01:35:01.128 7fbb3ce504a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.181 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '1'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '!='}]}, {'and_list': [{'operand1': 'ContrailConfig.elements.virtual_router_refs', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'ProuterData.tsn_agent_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}, {'operand1': 'ProuterData.gateway_mode', 'operand2': {'json_value': '"SERVER"'}, 'operation': '=='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-prouter-tsn-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-prouter-tsn-connectivity'], 'id_perms': {'created': '2025-09-01T01:35:01.130566', 'creator': None, 'description': 'Prouter connectivity to controlling ' 'tsn agent does not exist we look ' 'for non-empty value for ' 'tsn_agent_list', 'enable': True, 'last_modified': '2025-09-01T01:35:01.130566', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10504271381337467867, 'uuid_mslong': 3142675033853412581}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2b9d064a-ed52-44e5-91c6-ab2ad4ebebdb', 'uve_keys': {'uve_key': ['prouter']}}, 'oper': 'CREATE', 'request-id': 'req-253fdcfe-1152-4fe9-8a2b-7295bef192e3', 'type': 'alarm', 'uuid': '2b9d064a-ed52-44e5-91c6-ab2ad4ebebdb'} 09/01/2025 01:35:01.185 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:01.186 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:01.187 7fbb3ce504a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:01.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:01.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:01.189 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:01.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:01.191 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:01.192 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:01.193 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:01.194 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:01.195 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:01.196 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:01.196 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:01.197 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:01.198 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:01.199 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:01.199 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:01.200 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:01.201 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:01.202 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:01.202 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:01.203 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-storage-cluster-state'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-storage-cluster-state 09/01/2025 01:35:01.206 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-storage-cluster-state url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-storage-cluster-state"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["storage-cluster"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "StorageCluster.info_stats.status", "operation": "!=", "operand2": {"json_value": "0"}, "variables": ["StorageCluster.info_stats.health_summary"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Storage Cluster warning/errors.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-storage-cluster-state"}} domain = default-domain >> 09/01/2025 01:35:01.227 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'StorageCluster.info_stats.status', 'operand2': {'json_value': '0'}, 'operation': '!=', 'variables': ['StorageCluster.info_stats.health_summary']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-storage-cluster-state', 'fq_name': ['default-global-system-config', 'system-defined-storage-cluster-state'], 'id_perms': {'created': '2025-09-01T01:35:01.217585', 'creator': None, 'description': 'Storage Cluster warning/errors.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.217585', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12284578432857240966, 'uuid_mslong': 2852633633655180775}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '2796972d-0859-4de7-aa7b-96ff89e17586', 'uve_keys': {'uve_key': ['storage-cluster']}}, 'oper': 'CREATE', 'request-id': 'req-aa6e16b3-5471-478a-9bfe-77fcf366581b', 'type': 'alarm', 'uuid': '2796972d-0859-4de7-aa7b-96ff89e17586'} 09/01/2025 01:35:01.235 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-interface'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-interface 09/01/2025 01:35:01.238 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-interface url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-interface"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.down_interface_count", "operation": ">=", "operand2": {"json_value": "1"}, "variables": ["VrouterAgent.error_intf_list", "VrouterAgent.no_config_intf_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Vrouter interface(s) down.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-interface"}} domain = default-domain >> 09/01/2025 01:35:01.251 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace=contrail'] not found in DB: Unknown id: tag namespace=contrail 09/01/2025 01:35:01.252 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = namespace=contrail url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["namespace=contrail"], "tag_type_name": "namespace", "tag_value": "contrail", "name": "namespace=contrail", "display_name": "namespace=contrail", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:01.391 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.down_interface_count', 'operand2': {'json_value': '1'}, 'operation': '>=', 'variables': ['VrouterAgent.error_intf_list', 'VrouterAgent.no_config_intf_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-interface', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-interface'], 'id_perms': {'created': '2025-09-01T01:35:01.386361', 'creator': None, 'description': 'Vrouter interface(s) down.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.386361', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12208880729113631627, 'uuid_mslong': 17929068975300495646}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'f8d0d93d-a243-491e-a96e-a85673ee1b8b', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-9fc5f925-daa3-46cb-b392-b79e0fde1670', 'type': 'alarm', 'uuid': 'f8d0d93d-a243-491e-a96e-a85673ee1b8b'} 09/01/2025 01:35:01.509 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-xmpp-connectivity'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-xmpp-connectivity 09/01/2025 01:35:01.514 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-xmpp-connectivity url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-xmpp-connectivity"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["control-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "BgpRouterState.num_up_xmpp_peer", "operation": "==", "operand2": {"json_value": "null"}}]}, {"and_list": [{"operand1": "BgpRouterState.num_up_xmpp_peer", "operation": "!=", "operand2": {"uve_attribute": "BgpRouterState.num_xmpp_peer"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "XMPP peer mismatch.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-xmpp-connectivity"}} domain = default-domain >> 09/01/2025 01:35:01.601 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace=contrail'], 'obj_dict': {'display_name': 'namespace=contrail', 'fq_name': ['namespace=contrail'], 'id_perms': {'created': '2025-09-01T01:35:01.557786', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:01.557786', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9297889043599944578, 'uuid_mslong': 764438863256043651}}, 'name': 'namespace=contrail', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01010001', 'tag_type_name': 'namespace', 'tag_type_refs': [{'to': ['namespace'], 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}], 'tag_value': 'contrail', 'uuid': '0a9bd518-90e1-4083-8108-bcd2a4c56b82'}, 'oper': 'CREATE', 'request-id': 'req-386a6bfc-4958-462d-9f83-0968d86a25f4', 'type': 'tag', 'uuid': '0a9bd518-90e1-4083-8108-bcd2a4c56b82'} 09/01/2025 01:35:01.601 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:01.605 7fbb3cf6e0e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.605 7fbb3cf6e0e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = namespace=contrail elements = { display_name : "namespace=contrail"fq_name : ["namespace=contrail"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 764438863256043651, "uuid_lslong": 9297889043599944578}, "enable": true, "created": "2025-09-01T01:35:01.557786", "last_modified": "2025-09-01T01:35:01.557786", "description": null, "user_visible": true, "creator": null}name : "namespace=contrail"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01010001"tag_type_name : "namespace"tag_type_refs : [{"uuid": "884e5052-64fc-44a5-a883-82d91352fa8e", "to": ["namespace"]}]tag_value : "contrail"uuid : "0a9bd518-90e1-4083-8108-bcd2a4c56b82" } deleted = False >> 09/01/2025 01:35:01.605 7fbb3cf6e0e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.630 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:01.630 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:01.631 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:01.633 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:01.634 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:01.634 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:01.634 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:01.634 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:01.635 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:01.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:01.637 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:01.638 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:01.640 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:01.641 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:01.642 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:01.642 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:01.642 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:01.642 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:01.642 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:01.664 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'json_value': 'null'}, 'operation': '=='}]}, {'and_list': [{'operand1': 'BgpRouterState.num_up_xmpp_peer', 'operand2': {'uve_attribute': 'BgpRouterState.num_xmpp_peer'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-connectivity', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-connectivity'], 'id_perms': {'created': '2025-09-01T01:35:01.645266', 'creator': None, 'description': 'XMPP peer mismatch.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.645266', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12068159765698629765, 'uuid_mslong': 7356697359278424505}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '66183d45-53d8-41b9-a77a-b75df720d085', 'uve_keys': {'uve_key': ['control-node']}}, 'oper': 'CREATE', 'request-id': 'req-92fc4d7a-14e9-4658-a1ae-6a28b3678823', 'type': 'alarm', 'uuid': '66183d45-53d8-41b9-a77a-b75df720d085'} 09/01/2025 01:35:01.670 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-xmpp-close-reason'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-xmpp-close-reason 09/01/2025 01:35:01.680 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-xmpp-close-reason url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-xmpp-close-reason"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["xmpp-peer"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "XmppPeerInfoData.close_reason", "operation": "!=", "operand2": {"json_value": "null"}}, {"operand1": "XmppPeerInfoData.state_info.state", "operation": "!=", "operand2": {"json_value": "\"Established\""}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "XMPP connection closed towards peer, alarm has close reason", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-xmpp-close-reason"}} domain = default-domain >> 09/01/2025 01:35:01.689 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail'] not found in DB: Unknown id: project default-domain:k8s-contrail 09/01/2025 01:35:01.689 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = project identifier_name = default-domain:k8s-contrail url = http://10.0.0.50:8082/projects operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"project": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail"], "parent_type": "domain", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-contrail"}} domain = default-domain >> 09/01/2025 01:35:01.711 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'XmppPeerInfoData.close_reason', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'XmppPeerInfoData.state_info.state', 'operand2': {'json_value': '"Established"'}, 'operation': '!='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-xmpp-close-reason', 'fq_name': ['default-global-system-config', 'system-defined-xmpp-close-reason'], 'id_perms': {'created': '2025-09-01T01:35:01.691703', 'creator': None, 'description': 'XMPP connection closed towards ' 'peer, ' 'alarm has close reason', 'enable': True, 'last_modified': '2025-09-01T01:35:01.691703', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12165997184363612657, 'uuid_mslong': 4383944909124291530}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '3cd6e6aa-ef3c-4bca-a8d6-4dfb206549f1', 'uve_keys': {'uve_key': ['xmpp-peer']}}, 'oper': 'CREATE', 'request-id': 'req-b024a142-709f-492d-9da3-c5bfe05e542d', 'type': 'alarm', 'uuid': '3cd6e6aa-ef3c-4bca-a8d6-4dfb206549f1'} 09/01/2025 01:35:01.714 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-core-files'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-core-files 09/01/2025 01:35:01.720 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-core-files url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-core-files"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.all_core_file_list", "operand2": {"json_value": "null"}, "operation": "!="}, {"operand1": "NodeStatus.all_core_file_list", "operand2": {"json_value": "0"}, "operation": "size!="}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "A core file has been generated on the node.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-core-files"}} domain = default-domain >> 09/01/2025 01:35:01.727 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail'], 'obj_dict': {'display_name': 'k8s-contrail', 'fq_name': ['default-domain', 'k8s-contrail'], 'id_perms': {'created': '2025-09-01T01:35:01.717377', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:01.717377', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10022379383818498906, 'uuid_mslong': 581816361761063834}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a'}, 'oper': 'CREATE', 'request-id': 'req-f6923dc9-5523-457c-b447-e80fcbd284bc', 'type': 'project', 'uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a'} 09/01/2025 01:35:01.728 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:35:01.728 7fbb3cf6b0e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.728 7fbb3cf6b0e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-contrail elements = { display_name : "k8s-contrail"fq_name : ["default-domain", "k8s-contrail"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 581816361761063834, "uuid_lslong": 10022379383818498906}, "enable": true, "created": "2025-09-01T01:35:01.717377", "last_modified": "2025-09-01T01:35:01.717377", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "081306e5-ff47-439a-8b16-a4f2e0b3335a" } deleted = False >> 09/01/2025 01:35:01.728 7fbb3cf6b0e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.747 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'default-application-policy-set'] not found in DB: Unknown id: application_policy_set default-domain:k8s-contrail:default-application-policy-set 09/01/2025 01:35:01.766 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-domain:k8s-contrail:default-application-policy-set url = http://127.0.0.1/application-policy-sets operation = post body = {"application-policy-set": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "default-application-policy-set"], "parent_type": "project", "all_applications": true, "display_name": "default-application-policy-set", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:01.772 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': 'null'}, 'operation': '!='}, {'operand1': 'NodeStatus.all_core_file_list', 'operand2': {'json_value': '0'}, 'operation': 'size!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-core-files', 'fq_name': ['default-global-system-config', 'system-defined-core-files'], 'id_perms': {'created': '2025-09-01T01:35:01.757037', 'creator': None, 'description': 'A core file has been generated on ' 'the node.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.757037', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11501329277269896852, 'uuid_mslong': 7179776699111329448}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '63a3b0de-2528-4ea8-9f9c-ee0a8078ee94', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-a623d8e3-f755-46c3-a9fa-86f1648b62b1', 'type': 'alarm', 'uuid': '63a3b0de-2528-4ea8-9f9c-ee0a8078ee94'} 09/01/2025 01:35:01.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:01.797 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'display_name': 'default-application-policy-set', 'fq_name': ['default-domain', 'k8s-contrail', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:35:01.791455', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:01.791455', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11424969983852188087, 'uuid_mslong': 3187153430437121841}}, 'parent_type': 'project', 'parent_uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '2c3b0b28-a1df-4f31-9e8d-a5ab110291b7'}, 'oper': 'CREATE', 'request-id': 'req-f6923dc9-5523-457c-b447-e80fcbd284bc', 'type': 'application_policy_set', 'uuid': '2c3b0b28-a1df-4f31-9e8d-a5ab110291b7'} 09/01/2025 01:35:01.798 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:35:01.799 7fbb3ce50ea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.799 7fbb3ce50ea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-contrail:default-application-policy-set elements = { all_applications : truedisplay_name : "default-application-policy-set"fq_name : ["default-domain", "k8s-contrail", "default-application-policy-set"]id_perms : {"permissions": {"owner": "contrail-api", "owner_access": 7, "group": "admin", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 3187153430437121841, "uuid_lslong": 11424969983852188087}, "enable": true, "created": "2025-09-01T01:35:01.791455", "last_modified": "2025-09-01T01:35:01.791455", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "081306e5-ff47-439a-8b16-a4f2e0b3335a"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "2c3b0b28-a1df-4f31-9e8d-a5ab110291b7" } deleted = False >> 09/01/2025 01:35:01.799 7fbb3ce50ea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.816 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 081306e5-ff47-439a-8b16-a4f2e0b3335a object_type = project identifier_name = default-domain:k8s-contrail url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:01.819 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-pending-cassandra-compaction-tasks 09/01/2025 01:35:01.823 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-pending-cassandra-compaction-tasks url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-pending-cassandra-compaction-tasks"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["database-nodeconfig-database-node"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks", "operand2": {"json_value": "300"}, "operation": ">="}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Pending compaction tasks in cassandra crossed the configured threshold.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-pending-cassandra-compaction-tasks"}} domain = default-domain >> 09/01/2025 01:35:01.846 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail'], 'oper': 'UPDATE', 'request-id': 'req-f6923dc9-5523-457c-b447-e80fcbd284bc', 'type': 'project', 'uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a'} 09/01/2025 01:35:01.855 7fbb3ce50ea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.856 7fbb3ce50ea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-contrail elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-contrail", "default-application-policy-set"], "attr": null, "uuid": "2c3b0b28-a1df-4f31-9e8d-a5ab110291b7"}]display_name : "k8s-contrail"fq_name : ["default-domain", "k8s-contrail"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 581816361761063834, "uuid_lslong": 10022379383818498906}, "enable": true, "created": "2025-09-01T01:35:01.717377", "last_modified": "2025-09-01T01:35:01.824659", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "081306e5-ff47-439a-8b16-a4f2e0b3335a" } deleted = False >> 09/01/2025 01:35:01.858 7fbb3ce50ea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:01.872 7fbb3d0e3ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'default'] not found in DB: Unknown id: security_group default-domain:k8s-contrail:default 09/01/2025 01:35:01.873 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-contrail:default url = http://cn-jenkins-deploy-platform-juju-k8s-1454-1:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-api remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "default"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "d804800b-1a50-43a7-bbf4-00825bf2493d", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-contrail:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "bbd99afb-2454-4294-8e57-8389a1db15a8", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-contrail:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "faa114d9-5d7d-4d3e-887b-950566a31e78", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "8c6e98be-4225-4ef3-bec9-0b5e2d468aa6", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "default"}} domain = default-domain >> 09/01/2025 01:35:01.882 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-package-version-mismatch'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-package-version-mismatch 09/01/2025 01:35:01.892 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'CassandraStatusData.cassandra_compaction_task.pending_compaction_tasks', 'operand2': {'json_value': '300'}, 'operation': '>='}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-pending-cassandra-compaction-tasks', 'fq_name': ['default-global-system-config', 'system-defined-pending-cassandra-compaction-tasks'], 'id_perms': {'created': '2025-09-01T01:35:01.847333', 'creator': None, 'description': 'Pending compaction tasks in ' 'cassandra crossed the configured ' 'threshold.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.847333', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12785828124223818586, 'uuid_mslong': 1599492486595233073}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '163289f1-2c6a-4531-b170-62efcf8f4b5a', 'uve_keys': {'uve_key': ['database-nodeconfig-database-node']}}, 'oper': 'CREATE', 'request-id': 'req-45d7d2da-c7ae-4f84-9f98-ca6666f35674', 'type': 'alarm', 'uuid': '163289f1-2c6a-4531-b170-62efcf8f4b5a'} 09/01/2025 01:35:01.896 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-package-version-mismatch url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-package-version-mismatch"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["analytics-node", "analytics-alarm-node", "analytics-snmp-node", "config-node", "control-node", "database-node", "config-database-node", "vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "NodeStatus.running_package_version", "operation": "!=", "operand2": {"uve_attribute": "NodeStatus.installed_package_version"}}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "There is a mismatch between installed and running package version.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-package-version-mismatch"}} domain = default-domain >> 09/01/2025 01:35:01.935 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'NodeStatus.running_package_version', 'operand2': {'uve_attribute': 'NodeStatus.installed_package_version'}, 'operation': '!='}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-package-version-mismatch', 'fq_name': ['default-global-system-config', 'system-defined-package-version-mismatch'], 'id_perms': {'created': '2025-09-01T01:35:01.919901', 'creator': None, 'description': 'There is a mismatch between ' 'installed and running package ' 'version.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.919901', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9569444448425120023, 'uuid_mslong': 17242802562263173967}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'ef4abd92-f7e4-4b4f-84cd-7f0656369117', 'uve_keys': {'uve_key': ['analytics-node', 'analytics-alarm-node', 'analytics-snmp-node', 'config-node', 'control-node', 'database-node', 'config-database-node', 'vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-3cdce8d0-2fa6-434f-9793-383dff488d4f', 'type': 'alarm', 'uuid': 'ef4abd92-f7e4-4b4f-84cd-7f0656369117'} 09/01/2025 01:35:01.950 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'default'], 'obj_dict': {'display_name': 'default', 'fq_name': ['default-domain', 'k8s-contrail', 'default'], 'id_perms': {'created': '2025-09-01T01:35:01.920792', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:01.920792', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12418792943944759266, 'uuid_mslong': 12529932026945814864}}, 'parent_type': 'project', 'parent_uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'perms2': {'global_access': 0, 'owner': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'd804800b-1a50-43a7-bbf4-00825bf2493d', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-contrail:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'bbd99afb-2454-4294-8e57-8389a1db15a8', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-contrail:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'faa114d9-5d7d-4d3e-887b-950566a31e78', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '8c6e98be-4225-4ef3-bec9-0b5e2d468aa6', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000006, 'uuid': 'ade342ca-c5e5-4150-ac58-6a628af477e2'}, 'oper': 'CREATE', 'request-id': 'req-e8beff0e-d485-41cb-9653-01f21116aaf1', 'type': 'security_group', 'uuid': 'ade342ca-c5e5-4150-ac58-6a628af477e2'} 09/01/2025 01:35:01.954 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-limit-exceeded 09/01/2025 01:35:01.963 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-limit-exceeded url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-limit-exceeded"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 1, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.res_limit", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_limit_exceeded_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Agent resource usage exceeded configured watermark for resource.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-limit-exceeded"}} domain = default-domain >> 09/01/2025 01:35:01.969 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'default', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-contrail:default:ingress-access-control-list 09/01/2025 01:35:01.976 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-contrail:default:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "default", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000006", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "d804800b-1a50-43a7-bbf4-00825bf2493d", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000006", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "bbd99afb-2454-4294-8e57-8389a1db15a8", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:01.994 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 1, 'display_name': 'system-defined-vrouter-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:35:01.982979', 'creator': None, 'description': 'Agent resource usage exceeded ' 'configured watermark for resource.', 'enable': True, 'last_modified': '2025-09-01T01:35:01.982979', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11703449765111169089, 'uuid_mslong': 12023298412754059803}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a6db5634-48b9-421b-a26b-018dc8148041', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-bda61db9-3971-4fce-ada9-03bd9e2b9951', 'type': 'alarm', 'uuid': 'a6db5634-48b9-421b-a26b-018dc8148041'} 09/01/2025 01:35:01.999 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 081306e5-ff47-439a-8b16-a4f2e0b3335a object_type = project identifier_name = default-domain:k8s-contrail url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-contrail"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "contrail"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "contrail"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": "a28020d1-63f5-47e0-90e8-7a75e7378bf9"}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "isolated", "value": "False"}, "position": "isolated"}]} domain = default-domain >> 09/01/2025 01:35:02.014 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'default', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000006', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'd804800b-1a50-43a7-bbf4-00825bf2493d'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000006', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'bbd99afb-2454-4294-8e57-8389a1db15a8'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-contrail', 'default', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:02.004892', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:02.004892', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9548772609716880718, 'uuid_mslong': 14927954596232578009}}, 'parent_type': 'security-group', 'parent_uuid': 'ade342ca-c5e5-4150-ac58-6a628af477e2', 'perms2': {'global_access': 0, 'owner': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'owner_access': 7, 'share': []}, 'uuid': 'cf2abf9d-139a-43d9-8484-0e18d327854e'}, 'oper': 'CREATE', 'request-id': 'req-3bfe52a8-b579-41fa-9985-6b1ed0de9582', 'type': 'access_control_list', 'uuid': 'cf2abf9d-139a-43d9-8484-0e18d327854e'} 09/01/2025 01:35:02.025 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail'], 'oper': 'UPDATE', 'request-id': 'req-5414dd4a-0dbb-4323-bd25-63dcd92fad2d', 'type': 'project', 'uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a'} 09/01/2025 01:35:02.032 7fbb3cf5a720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.032 7fbb3cf5a720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-contrail elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "isolated", "value": "False"}, {"key": "k8s_uuid", "value": "a28020d1-63f5-47e0-90e8-7a75e7378bf9"}, {"key": "kind", "value": "Namespace"}, {"key": "name", "value": "contrail"}, {"key": "namespace", "value": "contrail"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-contrail"}]}application_policy_set_refs : [{"to": ["default-domain", "k8s-contrail", "default-application-policy-set"], "attr": null, "uuid": "2c3b0b28-a1df-4f31-9e8d-a5ab110291b7"}]display_name : "k8s-contrail"fq_name : ["default-domain", "k8s-contrail"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 581816361761063834, "uuid_lslong": 10022379383818498906}, "enable": true, "created": "2025-09-01T01:35:01.717377", "last_modified": "2025-09-01T01:35:02.002671", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "081306e5-ff47-439a-8b16-a4f2e0b3335a" } deleted = False >> 09/01/2025 01:35:02.032 7fbb3cf5a720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.049 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-table-limit-exceeded 09/01/2025 01:35:02.061 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'default', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-contrail:default:egress-access-control-list 09/01/2025 01:35:02.064 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-table-limit-exceeded url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-table-limit-exceeded"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.res_table_limit", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_limit_exceeded_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Agent resource usage exceeded table size for resource in vrouter.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-table-limit-exceeded"}} domain = default-domain >> 09/01/2025 01:35:02.066 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-contrail:default:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "default", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "faa114d9-5d7d-4d3e-887b-950566a31e78", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8c6e98be-4225-4ef3-bec9-0b5e2d468aa6", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:02.090 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.res_table_limit', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_limit_exceeded_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-table-limit-exceeded', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-table-limit-exceeded'], 'id_perms': {'created': '2025-09-01T01:35:02.077045', 'creator': None, 'description': 'Agent resource usage exceeded table ' 'size for resource in vrouter.', 'enable': True, 'last_modified': '2025-09-01T01:35:02.077045', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13572308632227373667, 'uuid_mslong': 6614552712968424813}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '5bcb9ca5-a494-4d6d-bc5a-86cab5e16263', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-5350116d-a370-45ea-a441-44398ce65320', 'type': 'alarm', 'uuid': '5bcb9ca5-a494-4d6d-bc5a-86cab5e16263'} 09/01/2025 01:35:02.093 7fbb3ceb7f40 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'] not found in DB: Unknown id: alarm default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:02.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:02.146 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:02.149 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:02.149 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:02.150 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:02.150 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:02.150 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:02.150 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:02.150 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:02.152 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = alarm identifier_name = default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down url = http://10.0.0.50:8082/alarms operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_alarm.py remote_ip = 10.0.0.50 body = {"alarm": {"uuid": null, "fq_name": ["default-global-system-config", "system-defined-vrouter-l3mh-physical-interface-link-down"], "parent_type": "global-system-config", "uve_keys": {"uve_key": ["vrouter"]}, "alarm_severity": 0, "alarm_rules": {"or_list": [{"and_list": [{"operand1": "VrouterAgent.l3mh_phy_interface_down", "operation": "==", "operand2": {"json_value": "true"}, "variables": ["VrouterAgent.vr_l3mh_intf_list"]}]}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "On l3 multihomed compute one or more physical interfaces are Inactive.", "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "system-defined-vrouter-l3mh-physical-interface-link-down"}} domain = default-domain >> 09/01/2025 01:35:02.158 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'default', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'faa114d9-5d7d-4d3e-887b-950566a31e78'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '8c6e98be-4225-4ef3-bec9-0b5e2d468aa6'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-contrail', 'default', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:02.094615', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:02.094615', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12448090761444631064, 'uuid_mslong': 3778302967327313713}}, 'parent_type': 'security-group', 'parent_uuid': 'ade342ca-c5e5-4150-ac58-6a628af477e2', 'perms2': {'global_access': 0, 'owner': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'owner_access': 7, 'share': []}, 'uuid': '346f3a87-cd59-4f31-acc0-80983fbaa218'}, 'oper': 'CREATE', 'request-id': 'req-9ee39db1-9e38-4b0d-846b-682b1482786b', 'type': 'access_control_list', 'uuid': '346f3a87-cd59-4f31-acc0-80983fbaa218'} 09/01/2025 01:35:02.180 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:02.180 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:02.181 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:02.181 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:02.181 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:02.182 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:02.182 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:02.182 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:02.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:02.184 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:02.247 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-network/d650e01f-39d4-4eb1-91f8-ca54373fcba6 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:02.249 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'obj_dict': {'alarm_rules': {'or_list': [{'and_list': [{'operand1': 'VrouterAgent.l3mh_phy_interface_down', 'operand2': {'json_value': 'true'}, 'operation': '==', 'variables': ['VrouterAgent.vr_l3mh_intf_list']}]}]}, 'alarm_severity': 0, 'display_name': 'system-defined-vrouter-l3mh-physical-interface-link-down', 'fq_name': ['default-global-system-config', 'system-defined-vrouter-l3mh-physical-interface-link-down'], 'id_perms': {'created': '2025-09-01T01:35:02.218693', 'creator': None, 'description': 'On l3 multihomed compute one or ' 'more physical interfaces are ' 'Inactive.', 'enable': True, 'last_modified': '2025-09-01T01:35:02.218693', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10910313226223040477, 'uuid_mslong': 10914525326012924032}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': '97782ef4-d501-4080-9769-3812ed4537dd', 'uve_keys': {'uve_key': ['vrouter']}}, 'oper': 'CREATE', 'request-id': 'req-41017a40-8c14-4dbc-adcf-aed02fca1acb', 'type': 'alarm', 'uuid': '97782ef4-d501-4080-9769-3812ed4537dd'} 09/01/2025 01:35:02.263 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-8c7c3952-7c75-4929-816b-276916e02baa', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:02.268 7fbb3ceb7f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.268 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:02.254547", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:02.268 7fbb3ceb7f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.280 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:02.351 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-8c7c3952-7c75-4929-816b-276916e02baa', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:02.357 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-network/a6c41605-29b5-4568-a40b-17e3f103c269 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:02.374 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-e158efc6-5d82-4115-87ed-1b5df6f86915', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:02.381 7fbb3ce50c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.381 7fbb3ce50c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:02.367612", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:02.381 7fbb3ce50c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.388 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:02.402 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-e158efc6-5d82-4115-87ed-1b5df6f86915', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:02.411 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg'] not found in DB: Unknown id: security_group default-domain:k8s-contrail:k8s-contrail-default-sg 09/01/2025 01:35:02.415 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-contrail:k8s-contrail-default-sg url = http://10.0.0.50:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "k8s-contrail-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "21fdef34-3bf5-42fa-96b1-22b0c5228755", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "31e479d7-1eaf-4aba-9e13-86ad6cd34c10", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "232937ba-3215-42cc-bea1-15bb5eeff8dd", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "cf245d60-d348-471f-b7e4-a265c7e2fc87", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-contrail-default-sg"}} domain = default-domain >> 09/01/2025 01:35:02.501 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg'], 'obj_dict': {'display_name': 'k8s-contrail-default-sg', 'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg'], 'id_perms': {'created': '2025-09-01T01:35:02.440898', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:02.440898', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10716835512065823629, 'uuid_mslong': 11418210244986487825}}, 'parent_type': 'project', 'parent_uuid': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '21fdef34-3bf5-42fa-96b1-22b0c5228755', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '31e479d7-1eaf-4aba-9e13-86ad6cd34c10', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '232937ba-3215-42cc-bea1-15bb5eeff8dd', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'cf245d60-d348-471f-b7e4-a265c7e2fc87', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000007, 'uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d'}, 'oper': 'CREATE', 'request-id': 'req-17a260cb-66df-4abe-aa94-0a775f7ca667', 'type': 'security_group', 'uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d'} 09/01/2025 01:35:02.536 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-contrail:k8s-contrail-default-sg:ingress-access-control-list 09/01/2025 01:35:02.538 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 87a687b0-9781-4994-aa87-dbc3f38997c0 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = http://10.0.0.50:8082/global-vrouter-config/87a687b0-9781-4994-aa87-dbc3f38997c0 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_encap.py remote_ip = 10.0.0.50 body = {"global-vrouter-config": {"uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "encapsulation_priorities": {"encapsulation": ["MPLSoUDP", "MPLSoGRE", "VXLAN"]}, "vxlan_network_identifier_mode": "automatic"}} domain = default-domain >> 09/01/2025 01:35:02.539 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9e75a1b8-b725-4411-94b9-d91e34f9938d object_type = security_group identifier_name = default-domain:k8s-contrail:k8s-contrail-default-sg url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "9e75a1b8-b725-4411-94b9-d91e34f9938d", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-contrail"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "contrail"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "k8s-contrail-default-sg"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:02.541 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-contrail:k8s-contrail-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "k8s-contrail-default-sg", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "21fdef34-3bf5-42fa-96b1-22b0c5228755", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "31e479d7-1eaf-4aba-9e13-86ad6cd34c10", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:02.551 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-f1e0bc91-b510-4c29-8f03-74b1320fe568', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:35:02.563 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-e7a802d7-8225-44cc-92df-e864f964ac58', 'type': 'security_group', 'uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d'} 09/01/2025 01:35:02.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: chown: 9e75a1b8-b725-4411-94b9-d91e34f9938d owner set to 081306e5-ff47-439a-8b16-a4f2e0b3335a 09/01/2025 01:35:02.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:02.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:02.597 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-3b559cce-98b5-4e6c-90a0-f1224781a986', 'type': 'security_group', 'uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d'} 09/01/2025 01:35:02.612 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '21fdef34-3bf5-42fa-96b1-22b0c5228755'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '31e479d7-1eaf-4aba-9e13-86ad6cd34c10'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:02.576967', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:02.576967', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13403080425643194522, 'uuid_mslong': 5016664225494878536}}, 'parent_type': 'security-group', 'parent_uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '459ec588-9e08-4948-ba01-4ea27274389a'}, 'oper': 'CREATE', 'request-id': 'req-efffa46b-2a53-4867-a5f2-a2c1d4c53658', 'type': 'access_control_list', 'uuid': '459ec588-9e08-4948-ba01-4ea27274389a'} 09/01/2025 01:35:02.631 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-contrail:k8s-contrail-default-sg:egress-access-control-list 09/01/2025 01:35:02.634 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-contrail:k8s-contrail-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-contrail", "k8s-contrail-default-sg", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "232937ba-3215-42cc-bea1-15bb5eeff8dd", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "cf245d60-d348-471f-b7e4-a265c7e2fc87", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:02.636 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:02.700 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:02.701 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:02.702 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:02.703 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:02.704 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:02.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:02.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:02.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:02.705 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:02.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:02.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:02.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:02.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:02.707 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:02.708 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:02.709 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:02.710 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:02.710 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:02.720 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '232937ba-3215-42cc-bea1-15bb5eeff8dd'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'cf245d60-d348-471f-b7e4-a265c7e2fc87'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-contrail', 'k8s-contrail-default-sg', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:02.652534', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:02.652534', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12395505328244393664, 'uuid_mslong': 16623783802522192305}}, 'parent_type': 'security-group', 'parent_uuid': '9e75a1b8-b725-4411-94b9-d91e34f9938d', 'perms2': {'global_access': 0, 'owner': '081306e5-ff47-439a-8b16-a4f2e0b3335a', 'owner_access': 7, 'share': []}, 'uuid': 'e6b38b4a-b43a-49b1-ac05-ae6bfd7806c0'}, 'oper': 'CREATE', 'request-id': 'req-86814ae2-c3de-44e8-a9c3-8d60164d274e', 'type': 'access_control_list', 'uuid': 'e6b38b4a-b43a-49b1-ac05-ae6bfd7806c0'} 09/01/2025 01:35:02.721 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-contrail-ingress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-contrail-ingress 09/01/2025 01:35:02.735 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-contrail-ingress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=contrail"], "tag_ids": [], "any": null}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-contrail-ingress"}} domain = default-domain >> 09/01/2025 01:35:02.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:02.800 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-contrail-ingress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-contrail-ingress', 'endpoint_1': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842753], 'tags': ['namespace=contrail'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-contrail-ingress'], 'id_perms': {'created': '2025-09-01T01:35:02.748868', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:02.748868', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10183644137596290291, 'uuid_mslong': 5575263468403573073}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=contrail'], 'uuid': '0a9bd518-90e1-4083-8108-bcd2a4c56b82'}], 'uuid': '4d5f5095-e368-4151-8d53-92634c6a84f3'}, 'oper': 'CREATE', 'request-id': 'req-7562bb48-db8d-4a88-81bf-f41930c935bd', 'type': 'firewall_rule', 'uuid': '4d5f5095-e368-4151-8d53-92634c6a84f3'} 09/01/2025 01:35:02.800 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:02.800 7fbb3cf5a680 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.801 7fbb3cf5a680 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-contrail-ingress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-contrail-ingress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=contrail"], "tag_ids": [16842753], "any": null}fq_name : ["default-policy-management", "k8s-Namespace-contrail-ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 5575263468403573073, "uuid_lslong": 10183644137596290291}, "enable": true, "created": "2025-09-01T01:35:02.748868", "last_modified": "2025-09-01T01:35:02.748868", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=contrail"], "uuid": "0a9bd518-90e1-4083-8108-bcd2a4c56b82"}]uuid : "4d5f5095-e368-4151-8d53-92634c6a84f3" } deleted = False >> 09/01/2025 01:35:02.801 7fbb3cf5a680 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.849 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:02.850 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:02.850 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:02.850 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:02.851 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:02.855 7fbb3cf5a680 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 09/01/2025 01:35:02.855 7fbb3cf5a680 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 09/01/2025 01:35:02.855 7fbb3cf5a680 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 09/01/2025 01:35:02.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_refs in resource firewall-policy: 'firewall_rule_refs' 09/01/2025 01:35:02.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:02.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:02.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:02.865 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:02.866 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:02.866 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:02.866 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:02.866 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:02.915 7fbb3cf5a680 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 87a687b0-9781-4994-aa87-dbc3f38997c0 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = http://10.0.0.50:8082/global-vrouter-config/87a687b0-9781-4994-aa87-dbc3f38997c0 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_global_vrouter_config.py remote_ip = 10.0.0.50 body = {"global-vrouter-config": {"uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "port_translation_pools": {"port_translation_pool": [{"protocol": "tcp", "port_range": {"start_port": 56000, "end_port": 57023}, "port_count": null}, {"protocol": "udp", "port_range": {"start_port": 57024, "end_port": 58047}, "port_count": null}]}}} domain = default-domain >> 09/01/2025 01:35:02.920 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3", "ref-fq-name": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "operation": "ADD", "attr": {"sequence": "00001.0"}} domain = default-domain >> 09/01/2025 01:35:02.934 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-2b864465-9bda-450c-9ea5-037acee3ce31', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:35:02.956 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-36286e64-75c0-4420-bcc6-90f5138fbab0', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:02.960 7fbb3ceb7f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.960 7fbb3ceb7f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:02.943843", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:02.960 7fbb3ceb7f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:02.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:02.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:03.023 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:03.023 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:03.023 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:03.023 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:03.023 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:03.024 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:03.024 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:03.087 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:03.088 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:03.089 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:03.090 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:03.091 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:03.092 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:03.093 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:03.094 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:03.095 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:03.096 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:03.102 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-contrail-egress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-contrail-egress 09/01/2025 01:35:03.107 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-contrail-egress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-contrail-egress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=contrail"], "tag_ids": [], "any": null}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-contrail-egress"}} domain = default-domain >> 09/01/2025 01:35:03.200 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-contrail-egress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-contrail-egress', 'endpoint_1': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842753], 'tags': ['namespace=contrail'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-contrail-egress'], 'id_perms': {'created': '2025-09-01T01:35:03.125206', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:03.125206', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11604150138806812443, 'uuid_mslong': 12778824617337704182}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=contrail'], 'uuid': '0a9bd518-90e1-4083-8108-bcd2a4c56b82'}], 'uuid': 'b1578148-cb6e-4ef6-a10a-3911dfe13f1b'}, 'oper': 'CREATE', 'request-id': 'req-32c4d195-9bb9-4cef-b82d-7a4b8a39c7af', 'type': 'firewall_rule', 'uuid': 'b1578148-cb6e-4ef6-a10a-3911dfe13f1b'} 09/01/2025 01:35:03.201 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:03.201 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.201 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-contrail-egress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-contrail-egress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=contrail"], "tag_ids": [16842753], "any": null}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}fq_name : ["default-policy-management", "k8s-Namespace-contrail-egress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12778824617337704182, "uuid_lslong": 11604150138806812443}, "enable": true, "created": "2025-09-01T01:35:03.125206", "last_modified": "2025-09-01T01:35:03.125206", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=contrail"], "uuid": "0a9bd518-90e1-4083-8108-bcd2a4c56b82"}]uuid : "b1578148-cb6e-4ef6-a10a-3911dfe13f1b" } deleted = False >> 09/01/2025 01:35:03.201 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.205 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:03.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:03.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:03.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:03.206 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:35:03.223 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:35:03.224 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:35:03.225 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:35:03.225 7fbb3cf5a900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:35:03.236 7fbb3cf5a900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 70eaa5ad-608f-40c9-a193-2be391842efd object_type = global_system_config identifier_name = default-global-system-config url = http://10.0.0.50:8082/global-system-config/70eaa5ad-608f-40c9-a193-2be391842efd operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_control.py remote_ip = 10.0.0.50 body = {"global-system-config": {"uuid": "70eaa5ad-608f-40c9-a193-2be391842efd", "autonomous_system": 64512, "ibgp_auto_mesh": true}} domain = default-domain >> 09/01/2025 01:35:03.249 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-57cf9ce7-8537-4ed5-a884-fe86d3d00ede', 'type': 'global_system_config', 'uuid': '70eaa5ad-608f-40c9-a193-2be391842efd'} 09/01/2025 01:35:03.281 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:03.281 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:03.281 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:03.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:03.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:03.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:03.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:03.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:03.302 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b", "ref-fq-name": ["default-policy-management", "k8s-Namespace-contrail-egress"], "operation": "ADD", "attr": {"sequence": "00002.0"}} domain = default-domain >> 09/01/2025 01:35:03.322 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-e5632fa9-9725-4744-86b1-f4682e824cfc', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:03.326 7fbb3cf5a900 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.326 7fbb3cf5a900 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:03.312771", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:03.326 7fbb3cf5a900 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.373 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s-svc=kubernetes'] not found in DB: Unknown id: tag k8s-svc=kubernetes 09/01/2025 01:35:03.374 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = k8s-svc=kubernetes url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["k8s-svc=kubernetes"], "tag_type_name": "k8s-svc", "tag_value": "kubernetes", "name": "k8s-svc=kubernetes", "display_name": "k8s-svc=kubernetes", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:03.381 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s-svc'] not found in DB: Unknown id: tag_type k8s-svc 09/01/2025 01:35:03.382 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag_type identifier_name = k8s-svc url = http://127.0.0.1/tag-types operation = post body = {"tag-type": {"uuid": null, "fq_name": ["k8s-svc"], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": null, "created": null, "last_modified": null, "description": null, "user_visible": false, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-svc"}} domain = default-domain >> 09/01/2025 01:35:03.396 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s-svc'], 'obj_dict': {'display_name': 'k8s-svc', 'fq_name': ['k8s-svc'], 'id_perms': {'created': '2025-09-01T01:35:03.390359', 'creator': None, 'description': None, 'enable': None, 'last_modified': '2025-09-01T01:35:03.390359', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': False, 'uuid': {'uuid_lslong': 11543464585065071630, 'uuid_mslong': 15268846470947556803}}, 'name': 'k8s-svc', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_type_id': '0x0103', 'uuid': 'd3e5d6f7-aece-4dc3-a032-9fe1561b040e'}, 'oper': 'CREATE', 'request-id': 'req-43818550-ab00-4cb8-8c28-dd5a32a8bb02', 'type': 'tag_type', 'uuid': 'd3e5d6f7-aece-4dc3-a032-9fe1561b040e'} 09/01/2025 01:35:03.430 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s-svc=kubernetes'], 'obj_dict': {'display_name': 'k8s-svc=kubernetes', 'fq_name': ['k8s-svc=kubernetes'], 'id_perms': {'created': '2025-09-01T01:35:03.423866', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:03.423866', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9232761476190750890, 'uuid_mslong': 7821842526510074911}}, 'name': 'k8s-svc=kubernetes', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01030000', 'tag_type_name': 'k8s-svc', 'tag_type_refs': [{'to': ['k8s-svc'], 'uuid': 'd3e5d6f7-aece-4dc3-a032-9fe1561b040e'}], 'tag_value': 'kubernetes', 'uuid': '6c8cc456-01ad-4c1f-8021-5ba53442c8aa'}, 'oper': 'CREATE', 'request-id': 'req-43818550-ab00-4cb8-8c28-dd5a32a8bb02', 'type': 'tag', 'uuid': '6c8cc456-01ad-4c1f-8021-5ba53442c8aa'} 09/01/2025 01:35:03.431 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:03.432 7fbb3cf5a400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.432 7fbb3cf5a400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = k8s-svc=kubernetes elements = { display_name : "k8s-svc=kubernetes"fq_name : ["k8s-svc=kubernetes"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7821842526510074911, "uuid_lslong": 9232761476190750890}, "enable": true, "created": "2025-09-01T01:35:03.423866", "last_modified": "2025-09-01T01:35:03.423866", "description": null, "user_visible": true, "creator": null}name : "k8s-svc=kubernetes"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01030000"tag_type_name : "k8s-svc"tag_type_refs : [{"uuid": "d3e5d6f7-aece-4dc3-a032-9fe1561b040e", "to": ["k8s-svc"]}]tag_value : "kubernetes"uuid : "6c8cc456-01ad-4c1f-8021-5ba53442c8aa" } deleted = False >> 09/01/2025 01:35:03.432 7fbb3cf5a400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:03.444 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:03.445 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:03.446 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:03.447 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:03.448 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:03.449 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:03.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:03.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:03.452 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:03.453 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:03.453 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:03.505 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:03.516 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:03.532 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-e519e413-4c52-4bd9-bb51-6e15fb5ebf86', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:35:03.534 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:03.534 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:03.534 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:03.535 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:03.536 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:03.537 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:03.537 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:03.539 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:03.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:03.541 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:03.542 7fbb3cf5a680 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.543 7fbb3cf5a680 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:35:03.521000", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:35:03.543 7fbb3cf5a680 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.591 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:03.617 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:03.618 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:03.628 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliances in resource service-appliance-set: 'service_appliances' 09/01/2025 01:35:03.628 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource service-appliance-set: 'tag_refs' 09/01/2025 01:35:03.628 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource service-appliance-set: 'service_template_back_refs' 09/01/2025 01:35:03.628 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource service-appliance-set: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:03.628 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource service-appliance-set: 'loadbalancer_back_refs' 09/01/2025 01:35:03.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'] not found in DB: Unknown id: virtual_machine_interface default-domain:k8s-default:k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 09/01/2025 01:35:03.689 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 09ad1cf5-8fd2-443e-9197-4d7e039e0029 not found in DB: Unknown id: 09ad1cf5-8fd2-443e-9197-4d7e039e0029 09/01/2025 01:35:03.703 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 09ad1cf5-8fd2-443e-9197-4d7e039e0029 object_type = virtual_machine_interface identifier_name = default-domain:k8s-default:k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 url = http://10.0.0.50:8082/virtual-machine-interfaces operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-machine-interface": {"uuid": "09ad1cf5-8fd2-443e-9197-4d7e039e0029", "fq_name": ["default-domain", "k8s-default", "k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"], "parent_type": "project", "port_security_enabled": true, "virtual_machine_interface_device_owner": "K8S:LOADBALANCER", "display_name": "default__kubernetes", "security_group_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-default-sg"]}], "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:03.767 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'obj_dict': {'display_name': 'default__kubernetes', 'fq_name': ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'id_perms': {'created': '2025-09-01T01:35:03.714936', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:03.714936', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10490939060604764201, 'uuid_mslong': 697245358317782078}}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'security_group_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-default-sg']}], 'uuid': '09ad1cf5-8fd2-443e-9197-4d7e039e0029', 'virtual_machine_interface_device_owner': 'K8S:LOADBALANCER', 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:09:ad:1c:f5:8f']}, 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'}]}, 'oper': 'CREATE', 'request-id': 'req-ebf64703-2423-41fb-8784-1254e6b22ab9', 'type': 'virtual_machine_interface', 'uuid': '09ad1cf5-8fd2-443e-9197-4d7e039e0029'} 09/01/2025 01:35:03.768 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVMITable:ContrailConfigTrace] map 09/01/2025 01:35:03.768 7fbb3cf5a9a0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.769 7fbb3cf5a9a0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 elements = { display_name : "default__kubernetes"fq_name : ["default-domain", "k8s-default", "k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 697245358317782078, "uuid_lslong": 10490939060604764201}, "enable": true, "created": "2025-09-01T01:35:03.714936", "last_modified": "2025-09-01T01:35:03.714936", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truesecurity_group_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-default-sg"]}]uuid : "09ad1cf5-8fd2-443e-9197-4d7e039e0029"virtual_machine_interface_device_owner : "K8S:LOADBALANCER"virtual_machine_interface_mac_addresses : {"mac_address": ["02:09:ad:1c:f5:8f"]}virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}] } deleted = False >> 09/01/2025 01:35:03.769 7fbb3cf5a9a0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:03.784 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 09ad1cf5-8fd2-443e-9197-4d7e039e0029 object_type = virtual_machine_interface identifier_name = default-domain:k8s-default:k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:03.792 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'oper': 'UPDATE', 'request-id': 'req-ebf64703-2423-41fb-8784-1254e6b22ab9', 'type': 'virtual_machine_interface', 'uuid': '09ad1cf5-8fd2-443e-9197-4d7e039e0029'} 09/01/2025 01:35:03.800 7fbb3cf5a860 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.800 7fbb3cf5a860 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 elements = { display_name : "default__kubernetes"fq_name : ["default-domain", "k8s-default", "k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 697245358317782078, "uuid_lslong": 10490939060604764201}, "enable": true, "created": "2025-09-01T01:35:03.714936", "last_modified": "2025-09-01T01:35:03.788042", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network", "k8s-default-service-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "a099888b-9d11-4587-8b98-4d1673309c9e"}]security_group_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-default-sg"], "attr": null, "uuid": "4de733ab-781c-4219-bcab-efd8adb97fdf"}]uuid : "09ad1cf5-8fd2-443e-9197-4d7e039e0029"virtual_machine_interface_device_owner : "K8S:LOADBALANCER"virtual_machine_interface_mac_addresses : {"mac_address": ["02:09:ad:1c:f5:8f"]}virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "attr": null, "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}] } deleted = False >> 09/01/2025 01:35:03.800 7fbb3cf5a860 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:03.800 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:03.801 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:03.801 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-machine-interface: 'virtual_machine_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:03.802 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:03.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:03.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:03.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:03.851 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23'] not found in DB: Unknown id: instance_ip kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23 09/01/2025 01:35:03.852 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 7a7d590e-3340-4e88-a363-525aacd0cc23 not found in DB: Unknown id: 7a7d590e-3340-4e88-a363-525aacd0cc23 09/01/2025 01:35:03.853 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 7a7d590e-3340-4e88-a363-525aacd0cc23 object_type = instance_ip identifier_name = kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23 url = http://10.0.0.50:8082/instance-ips operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"instance-ip": {"uuid": "7a7d590e-3340-4e88-a363-525aacd0cc23", "fq_name": ["kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23"], "instance_ip_address": "10.96.0.1", "subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400", "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kubernetes", "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}], "virtual_machine_interface_refs": [{"to": ["default-domain", "k8s-default", "k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"], "uuid": "09ad1cf5-8fd2-443e-9197-4d7e039e0029"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:03.877 7fbb3ceb7400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: AddrMgmt: alloc IIP 10.96.0.1 for vn=['default-domain', 'k8s-default', 'k8s-default-service-network'], tenant=default-project, askip=10.96.0.1 09/01/2025 01:35:03.903 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23'], 'obj_dict': {'display_name': 'kubernetes', 'fq_name': ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23'], 'id_perms': {'created': '2025-09-01T01:35:03.879882', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:03.879882', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11773344400276507683, 'uuid_mslong': 8826308762240306824}}, 'instance_ip_address': '10.96.0.1', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'subnet_uuid': 'cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400', 'uuid': '7a7d590e-3340-4e88-a363-525aacd0cc23', 'virtual_machine_interface_refs': [{'to': ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'uuid': '09ad1cf5-8fd2-443e-9197-4d7e039e0029'}], 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'}]}, 'oper': 'CREATE', 'request-id': 'req-1d81df02-2873-48de-a3e5-27ed63c314a4', 'type': 'instance_ip', 'uuid': '7a7d590e-3340-4e88-a363-525aacd0cc23'} 09/01/2025 01:35:03.912 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:03.912 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:03.912 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:03.913 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:03.919 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'] not found in DB: Unknown id: loadbalancer default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 09/01/2025 01:35:03.923 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 4149e3a7-961c-4071-90b8-b63acefd96b1 not found in DB: Unknown id: 4149e3a7-961c-4071-90b8-b63acefd96b1 09/01/2025 01:35:03.924 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 4149e3a7-961c-4071-90b8-b63acefd96b1 object_type = loadbalancer identifier_name = default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 url = http://10.0.0.50:8082/loadbalancers operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer": {"uuid": "4149e3a7-961c-4071-90b8-b63acefd96b1", "fq_name": ["default-domain", "k8s-default", "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"], "parent_type": "project", "loadbalancer_properties": {"status": null, "provisioning_status": "ACTIVE", "operating_status": "ONLINE", "vip_subnet_id": null, "vip_address": "10.96.0.1", "admin_state": true}, "loadbalancer_provider": "native", "display_name": "default__kubernetes", "service_appliance_set_refs": [{"to": ["default-global-system-config", "native"], "uuid": "ce9684c7-029d-4e16-ad7f-6b14554d8775"}], "virtual_machine_interface_refs": [{"to": ["default-domain", "k8s-default", "k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"], "uuid": "09ad1cf5-8fd2-443e-9197-4d7e039e0029"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:04.001 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'obj_dict': {'display_name': 'default__kubernetes', 'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'id_perms': {'created': '2025-09-01T01:35:03.939097', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:03.939097', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10428285300873467569, 'uuid_mslong': 4704541594682671217}}, 'loadbalancer_properties': {'admin_state': True, 'operating_status': 'ONLINE', 'provisioning_status': 'ACTIVE', 'status': None, 'vip_address': '10.96.0.1', 'vip_subnet_id': None}, 'loadbalancer_provider': 'native', 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'service_appliance_set_refs': [{'to': ['default-global-system-config', 'native'], 'uuid': 'ce9684c7-029d-4e16-ad7f-6b14554d8775'}], 'uuid': '4149e3a7-961c-4071-90b8-b63acefd96b1', 'virtual_machine_interface_refs': [{'to': ['default-domain', 'k8s-default', 'k8s__Service__kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'uuid': '09ad1cf5-8fd2-443e-9197-4d7e039e0029'}]}, 'oper': 'CREATE', 'request-id': 'req-0be6ff65-919f-4513-9db1-ad436c8d62fb', 'type': 'loadbalancer', 'uuid': '4149e3a7-961c-4071-90b8-b63acefd96b1'} 09/01/2025 01:35:04.050 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 4149e3a7-961c-4071-90b8-b63acefd96b1 object_type = loadbalancer identifier_name = default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1 url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "4149e3a7-961c-4071-90b8-b63acefd96b1", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Service"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-default"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "default"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "kubernetes"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:04.067 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'oper': 'UPDATE', 'request-id': 'req-3ba112e4-6a8f-4de0-b8d7-a0443d433b47', 'type': 'loadbalancer', 'uuid': '4149e3a7-961c-4071-90b8-b63acefd96b1'} 09/01/2025 01:35:04.074 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:04.081 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:04.089 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-c4362afb-39c1-4736-8e15-c1eb7f6b633f', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:35:04.095 7fbb3ce50ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.116 7fbb3ce50ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:35:04.084588", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:35:04.116 7fbb3ce50ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:04.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:04.118 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:04.119 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:04.131 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:04.136 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:04.147 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-c077d094-eca3-4f1c-aa1d-8f898a4dd81c', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:35:04.159 7fbb3ce50ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.159 7fbb3ce50ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:35:04.139951", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:35:04.159 7fbb3ce50ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.160 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:04.161 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:04.162 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:04.163 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:04.164 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:04.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:04.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:04.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_refs in resource loadbalancer: 'service_instance_refs' 09/01/2025 01:35:04.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer: 'tag_refs' 09/01/2025 01:35:04.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource loadbalancer: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:04.182 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23', '6e324ad2-40d0-4016-9315-3daed4aa8660'] not found in DB: Unknown id: floating_ip kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23:6e324ad2-40d0-4016-9315-3daed4aa8660 09/01/2025 01:35:04.183 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'] not found in DB: Unknown id: loadbalancer_listener default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529 09/01/2025 01:35:04.189 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 6e324ad2-40d0-4016-9315-3daed4aa8660 not found in DB: Unknown id: 6e324ad2-40d0-4016-9315-3daed4aa8660 09/01/2025 01:35:04.189 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 6e324ad2-40d0-4016-9315-3daed4aa8660 object_type = floating_ip identifier_name = kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23:6e324ad2-40d0-4016-9315-3daed4aa8660 url = http://10.0.0.50:8082/floating-ips operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"floating-ip": {"uuid": "6e324ad2-40d0-4016-9315-3daed4aa8660", "fq_name": ["kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23", "6e324ad2-40d0-4016-9315-3daed4aa8660"], "parent_type": "instance-ip", "floating_ip_address": "10.96.0.1", "floating_ip_traffic_direction": "ingress", "project_refs": [{"to": ["default-domain", "k8s-default"], "uuid": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "9023d5fc-34ad-4e7e-9d9e-872a06007bf8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "6e324ad2-40d0-4016-9315-3daed4aa8660"}} domain = default-domain >> 09/01/2025 01:35:04.190 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 11d02361-e609-4206-93f6-a60460b35529 not found in DB: Unknown id: 11d02361-e609-4206-93f6-a60460b35529 09/01/2025 01:35:04.190 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 11d02361-e609-4206-93f6-a60460b35529 object_type = loadbalancer_listener identifier_name = default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529 url = http://10.0.0.50:8082/loadbalancer-listeners operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-listener": {"uuid": "11d02361-e609-4206-93f6-a60460b35529", "fq_name": ["default-domain", "k8s-default", "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529"], "parent_type": "project", "loadbalancer_listener_properties": {"protocol": "TCP", "protocol_port": 443, "admin_state": true, "connection_limit": null, "default_tls_container": null, "sni_containers": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529", "loadbalancer_refs": [{"to": ["default-domain", "k8s-default", "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1"], "uuid": "4149e3a7-961c-4071-90b8-b63acefd96b1"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:04.207 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23', '6e324ad2-40d0-4016-9315-3daed4aa8660'], 'obj_dict': {'display_name': '6e324ad2-40d0-4016-9315-3daed4aa8660', 'floating_ip_address': '10.96.0.1', 'floating_ip_traffic_direction': 'ingress', 'fq_name': ['kubernetes__7a7d590e-3340-4e88-a363-525aacd0cc23', '6e324ad2-40d0-4016-9315-3daed4aa8660'], 'id_perms': {'created': '2025-09-01T01:35:04.197352', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:04.197352', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10598445119187879520, 'uuid_mslong': 7940491359898583062}}, 'parent_type': 'instance-ip', 'parent_uuid': '7a7d590e-3340-4e88-a363-525aacd0cc23', 'perms2': {'global_access': 0, 'owner': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'owner_access': 7, 'share': []}, 'project_refs': [{'to': ['default-domain', 'k8s-default'], 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'}], 'uuid': '6e324ad2-40d0-4016-9315-3daed4aa8660'}, 'oper': 'CREATE', 'request-id': 'req-6ed18d80-3769-462c-a82c-86b3504280a7', 'type': 'floating_ip', 'uuid': '6e324ad2-40d0-4016-9315-3daed4aa8660'} 09/01/2025 01:35:04.212 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'obj_dict': {'display_name': 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529', 'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'id_perms': {'created': '2025-09-01T01:35:04.198842', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:04.198842', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10661891705578673449, 'uuid_mslong': 1283564797178757638}}, 'loadbalancer_listener_properties': {'admin_state': True, 'connection_limit': None, 'default_tls_container': None, 'protocol': 'TCP', 'protocol_port': 443, 'sni_containers': []}, 'loadbalancer_refs': [{'to': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1'], 'uuid': '4149e3a7-961c-4071-90b8-b63acefd96b1'}], 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '11d02361-e609-4206-93f6-a60460b35529'}, 'oper': 'CREATE', 'request-id': 'req-39ff537b-90f1-4945-9ff4-d0ea178df0cc', 'type': 'loadbalancer_listener', 'uuid': '11d02361-e609-4206-93f6-a60460b35529'} 09/01/2025 01:35:04.218 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 11d02361-e609-4206-93f6-a60460b35529 object_type = loadbalancer_listener identifier_name = default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529 url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "11d02361-e609-4206-93f6-a60460b35529", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "portName", "value": "https"}, "position": "portName"}]} domain = default-domain >> 09/01/2025 01:35:04.225 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'oper': 'UPDATE', 'request-id': 'req-d62472a7-40d7-4304-961e-f17fc8ce3fb8', 'type': 'loadbalancer_listener', 'uuid': '11d02361-e609-4206-93f6-a60460b35529'} 09/01/2025 01:35:04.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:04.279 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:04.289 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-667400c6-66ca-45e3-a5f8-7645de4c80d1', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:35:04.291 7fbb3ce50ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.291 7fbb3ce50ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:35:04.283672", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:35:04.291 7fbb3ce50ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:04.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:04.293 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:04.294 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:04.299 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:04.299 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:04.304 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_refs in resource loadbalancer-listener: 'loadbalancer_refs' 09/01/2025 01:35:04.304 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:04.304 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:04.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'] not found in DB: Unknown id: loadbalancer_pool default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529 09/01/2025 01:35:04.368 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = loadbalancer_pool identifier_name = default-domain:k8s-default:kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529 url = http://10.0.0.50:8082/loadbalancer-pools operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-pool": {"uuid": null, "fq_name": ["default-domain", "k8s-default", "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529"], "parent_type": "project", "loadbalancer_pool_properties": {"status": null, "status_description": null, "admin_state": true, "protocol": "TCP", "loadbalancer_method": null, "subnet_id": null, "session_persistence": null, "persistence_cookie_name": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "loadbalancer_listener_refs": [{"to": ["default-domain", "k8s-default", "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529"], "uuid": "11d02361-e609-4206-93f6-a60460b35529"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529"}} domain = default-domain >> 09/01/2025 01:35:04.386 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'obj_dict': {'display_name': 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529', 'fq_name': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'id_perms': {'created': '2025-09-01T01:35:04.378765', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:04.378765', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11940756889999979924, 'uuid_mslong': 47868323585213139}}, 'loadbalancer_listener_refs': [{'to': ['default-domain', 'k8s-default', 'kubernetes__4149e3a7-961c-4071-90b8-b63acefd96b1-TCP-443-11d02361-e609-4206-93f6-a60460b35529'], 'uuid': '11d02361-e609-4206-93f6-a60460b35529'}], 'loadbalancer_pool_properties': {'admin_state': True, 'loadbalancer_method': None, 'persistence_cookie_name': None, 'protocol': 'TCP', 'session_persistence': None, 'status': None, 'status_description': None, 'subnet_id': None}, 'parent_type': 'project', 'parent_uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '00aa0ffc-974a-42d3-a5b6-172069bf3594'}, 'oper': 'CREATE', 'request-id': 'req-ae91cf19-b028-4cc5-a49d-8a7f13fc7c9e', 'type': 'loadbalancer_pool', 'uuid': '00aa0ffc-974a-42d3-a5b6-172069bf3594'} 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:04.398 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:04.399 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource global-vrouter-config: 'security_logging_objects' 09/01/2025 01:35:04.399 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-vrouter-config: 'tag_refs' 09/01/2025 01:35:04.399 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource global-vrouter-config: 'application_policy_set_back_refs' 09/01/2025 01:35:04.405 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource instance-ip: 'virtual_network_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource instance-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:04.406 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:04.412 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource floating-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:04.412 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource floating-ip: 'tag_refs' 09/01/2025 01:35:04.412 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource floating-ip: 'customer_attachment_back_refs' 09/01/2025 01:35:04.451 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 87a687b0-9781-4994-aa87-dbc3f38997c0 object_type = global_vrouter_config identifier_name = default-global-system-config:default-global-vrouter-config url = http://10.0.0.50:8082/global-vrouter-config/87a687b0-9781-4994-aa87-dbc3f38997c0 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"global-vrouter-config": {"uuid": "87a687b0-9781-4994-aa87-dbc3f38997c0", "fq_name": ["default-global-system-config", "default-global-vrouter-config"], "parent_type": "global-system-config", "linklocal_services": {"linklocal_service_entry": [{"linklocal_service_name": "metadata", "linklocal_service_ip": "169.254.169.254", "linklocal_service_port": 80, "ip_fabric_DNS_service_name": "", "ip_fabric_service_port": 8775, "ip_fabric_service_ip": ["127.0.0.1"]}, {"linklocal_service_name": "default-domain-k8s-default-kubernetes-443", "linklocal_service_ip": "10.96.0.1", "linklocal_service_port": 443, "ip_fabric_DNS_service_name": "", "ip_fabric_service_port": 6443, "ip_fabric_service_ip": ["10.0.0.50"]}]}}} domain = default-domain >> 09/01/2025 01:35:04.465 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'default-global-vrouter-config'], 'oper': 'UPDATE', 'request-id': 'req-b9fd73f9-712b-4ccf-b991-429312f9ebdc', 'type': 'global_vrouter_config', 'uuid': '87a687b0-9781-4994-aa87-dbc3f38997c0'} 09/01/2025 01:35:04.488 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name=default'] not found in DB: Unknown id: tag kubernetes.io__slash__metadata.name=default 09/01/2025 01:35:04.489 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = kubernetes.io__slash__metadata.name=default url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name=default"], "tag_type_name": "kubernetes.io__slash__metadata.name", "tag_value": "default", "name": "kubernetes.io__slash__metadata.name=default", "display_name": "kubernetes.io__slash__metadata.name=default", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:04.503 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name=default'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name=default', 'fq_name': ['kubernetes.io__slash__metadata.name=default'], 'id_perms': {'created': '2025-09-01T01:35:04.498091', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:04.498091', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13662651753165036361, 'uuid_mslong': 1818953846815737330}}, 'name': 'kubernetes.io__slash__metadata.name=default', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01020001', 'tag_type_name': 'kubernetes.io__slash__metadata.name', 'tag_type_refs': [{'to': ['kubernetes.io__slash__metadata.name'], 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}], 'tag_value': 'default', 'uuid': '193e38e2-ee48-45f2-bd9b-7d61b5451b49'}, 'oper': 'CREATE', 'request-id': 'req-1eb9ae72-3722-444e-bbec-f19cadc5c2a6', 'type': 'tag', 'uuid': '193e38e2-ee48-45f2-bd9b-7d61b5451b49'} 09/01/2025 01:35:04.504 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:04.505 7fbb3cf5ac20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.505 7fbb3cf5ac20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = kubernetes.io__slash__metadata.name=default elements = { display_name : "kubernetes.io__slash__metadata.name=default"fq_name : ["kubernetes.io__slash__metadata.name=default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1818953846815737330, "uuid_lslong": 13662651753165036361}, "enable": true, "created": "2025-09-01T01:35:04.498091", "last_modified": "2025-09-01T01:35:04.498091", "description": null, "user_visible": true, "creator": null}name : "kubernetes.io__slash__metadata.name=default"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01020001"tag_type_name : "kubernetes.io__slash__metadata.name"tag_type_refs : [{"uuid": "c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6", "to": ["kubernetes.io__slash__metadata.name"]}]tag_value : "default"uuid : "193e38e2-ee48-45f2-bd9b-7d61b5451b49" } deleted = False >> 09/01/2025 01:35:04.505 7fbb3cf5ac20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:04.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:04.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:04.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:04.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:04.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:04.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:04.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:04.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:04.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:04.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:04.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:04.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:04.570 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:04.570 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:04.570 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:04.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace=default'] not found in DB: Unknown id: tag namespace=default 09/01/2025 01:35:04.576 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = namespace=default url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["namespace=default"], "tag_type_name": "namespace", "tag_value": "default", "name": "namespace=default", "display_name": "namespace=default", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:04.610 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace=default'], 'obj_dict': {'display_name': 'namespace=default', 'fq_name': ['namespace=default'], 'id_perms': {'created': '2025-09-01T01:35:04.589424', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:04.589424', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10845566140282948963, 'uuid_mslong': 18086324725812776608}}, 'name': 'namespace=default', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01010002', 'tag_type_name': 'namespace', 'tag_type_refs': [{'to': ['namespace'], 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}], 'tag_value': 'default', 'uuid': 'faff8883-3e6f-4ea0-9683-30f13b91cd63'}, 'oper': 'CREATE', 'request-id': 'req-423e7e90-c62d-4292-a7c5-f57bb4a9127d', 'type': 'tag', 'uuid': 'faff8883-3e6f-4ea0-9683-30f13b91cd63'} 09/01/2025 01:35:04.610 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:04.611 7fbb3cf5acc0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.612 7fbb3cf5acc0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = namespace=default elements = { display_name : "namespace=default"fq_name : ["namespace=default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 18086324725812776608, "uuid_lslong": 10845566140282948963}, "enable": true, "created": "2025-09-01T01:35:04.589424", "last_modified": "2025-09-01T01:35:04.589424", "description": null, "user_visible": true, "creator": null}name : "namespace=default"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01010002"tag_type_name : "namespace"tag_type_refs : [{"uuid": "884e5052-64fc-44a5-a883-82d91352fa8e", "to": ["namespace"]}]tag_value : "default"uuid : "faff8883-3e6f-4ea0-9683-30f13b91cd63" } deleted = False >> 09/01/2025 01:35:04.612 7fbb3cf5acc0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.654 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:04.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:04.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:04.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:04.658 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:04.658 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:04.658 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:04.658 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:04.658 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:04.659 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:04.660 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:04.661 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:04.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:04.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:04.664 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:04.664 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:04.664 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:04.664 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:04.664 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:04.718 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:04.722 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 9023d5fc-34ad-4e7e-9d9e-872a06007bf8 object_type = project identifier_name = default-domain:k8s-default url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:04.736 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default'], 'oper': 'UPDATE', 'request-id': 'req-7e6f9af7-71b2-4c01-bfe5-c70b2589413b', 'type': 'project', 'uuid': '9023d5fc-34ad-4e7e-9d9e-872a06007bf8'} 09/01/2025 01:35:04.741 7fbb3cf5a180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.741 7fbb3cf5a180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-default", "default-application-policy-set"], "attr": null, "uuid": "f33ac37d-3499-4c5d-b303-467269a463d2"}]display_name : "k8s-default"fq_name : ["default-domain", "k8s-default"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10386380444838743678, "uuid_lslong": 11357663924834040824}, "enable": true, "created": "2025-09-01T01:34:55.387855", "last_modified": "2025-09-01T01:35:04.730646", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8" } deleted = False >> 09/01/2025 01:35:04.741 7fbb3cf5a180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:04.742 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:04.743 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:04.744 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:04.745 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:04.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:04.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 09/01/2025 01:35:04.795 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 09/01/2025 01:35:04.795 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 09/01/2025 01:35:04.795 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 09/01/2025 01:35:04.795 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 09/01/2025 01:35:04.803 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:04.803 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:04.803 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:04.803 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:04.804 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:04.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:04.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:04.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:04.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:04.805 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:04.851 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-network/d650e01f-39d4-4eb1-91f8-ca54373fcba6 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "fabric_snat": true}} domain = default-domain >> 09/01/2025 01:35:04.861 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-96f09b0e-4843-46a4-bcc3-14d7074bee54', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:04.886 7fbb3cf5aa40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.886 7fbb3cf5aa40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:04.856193", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:04.886 7fbb3cf5aa40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.892 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:04.900 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-96f09b0e-4843-46a4-bcc3-14d7074bee54', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:04.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:04.915 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:04.916 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:04.924 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://127.0.0.1/virtual-networks operation = http_put domain = default-domain >> 09/01/2025 01:35:04.934 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-11dc5953-d3ad-42b5-ba91-597bc14c0006', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:04.937 7fbb3cf5ac20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.937 7fbb3cf5ac20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:04.928527", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:04.937 7fbb3cf5ac20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:04.949 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:04.964 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-11dc5953-d3ad-42b5-ba91-597bc14c0006', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:05.015 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_refs in resource network-ipam: 'virtual_DNS_refs' 09/01/2025 01:35:05.016 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-ipam: 'tag_refs' 09/01/2025 01:35:05.016 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-ipam: 'virtual_network_back_refs' 09/01/2025 01:35:05.016 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource network-ipam: 'virtual_router_back_refs' 09/01/2025 01:35:05.016 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource network-ipam: 'instance_ip_back_refs' 09/01/2025 01:35:05.074 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:05.074 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:05.075 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:05.076 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:05.082 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-network/a6c41605-29b5-4568-a40b-17e3f103c269 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "fabric_snat": false}} domain = default-domain >> 09/01/2025 01:35:05.098 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-8b5248da-cf34-4c23-99f8-c358e451cdee', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:05.100 7fbb3cf5acc0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.100 7fbb3cf5acc0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:05.093971", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:05.100 7fbb3cf5acc0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.109 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:05.120 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-8b5248da-cf34-4c23-99f8-c358e451cdee', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:05.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:05.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:05.138 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:05.138 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:05.141 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:05.141 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:05.142 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:05.142 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:05.142 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:05.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:05.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:05.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:05.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:05.145 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:05.186 7fbb3cf5ad60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:05.187 7fbb3cf5ad60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:05.188 7fbb3cf5ad60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.198 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://127.0.0.1/virtual-networks operation = http_put domain = default-domain >> 09/01/2025 01:35:05.219 7fbb3cf5aa40 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:35:05.220 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-532a29a8-81ca-4ca6-afe9-0d0a1bd27472', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:05.224 7fbb3cf5aa40 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:35:05.230 7fbb3cf5aa40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.230 7fbb3cf5aa40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:05.206021", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:05.230 7fbb3cf5aa40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.239 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:05.245 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-532a29a8-81ca-4ca6-afe9-0d0a1bd27472', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:05.264 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-policy: 'tag_refs' 09/01/2025 01:35:05.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource network-policy: 'security_logging_object_back_refs' 09/01/2025 01:35:05.265 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-policy: 'virtual_network_back_refs' 09/01/2025 01:35:05.316 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-policy: 'tag_refs' 09/01/2025 01:35:05.316 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource network-policy: 'security_logging_object_back_refs' 09/01/2025 01:35:05.316 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-policy: 'virtual_network_back_refs' 09/01/2025 01:35:05.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource network-policy: 'tag_refs' 09/01/2025 01:35:05.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource network-policy: 'security_logging_object_back_refs' 09/01/2025 01:35:05.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource network-policy: 'virtual_network_back_refs' 09/01/2025 01:35:05.382 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 1f1a62da-5984-46dc-82ee-e07966ea5188 object_type = network_policy identifier_name = default-domain:k8s-default:k8s-default-pod-service-np url = http://10.0.0.50:8082/network-policy/1f1a62da-5984-46dc-82ee-e07966ea5188 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"network-policy": {"uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188", "fq_name": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "parent_type": "project"}} domain = default-domain >> 09/01/2025 01:35:05.422 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-service-np'], 'oper': 'UPDATE', 'request-id': 'req-f4014d9f-0e90-4d56-878d-92b1d34140c3', 'type': 'network_policy', 'uuid': '1f1a62da-5984-46dc-82ee-e07966ea5188'} 09/01/2025 01:35:05.426 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-network", "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-type": "network-policy", "ref-uuid": "a6209b87-6082-4775-b38a-86ce0eab279a", "ref-fq-name": ["default-domain", "k8s-default", "k8s-default-service-np"], "operation": "ADD", "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}} domain = default-domain >> 09/01/2025 01:35:05.439 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-a969ab3e-d4dd-4022-bdbc-9a1f72472875', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:05.445 7fbb3cf5aea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.445 7fbb3cf5aea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:05.433296", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:05.445 7fbb3cf5aea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:05.449 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:05.459 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-a969ab3e-d4dd-4022-bdbc-9a1f72472875', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:05.619 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"} domain = default-domain >> 09/01/2025 01:35:05.624 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"} domain = default-domain >> 09/01/2025 01:35:05.628 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "ref-uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"} domain = default-domain >> 09/01/2025 01:35:05.639 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"} domain = default-domain >> 09/01/2025 01:35:05.658 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/ref-relax-for-delete operation = ref-relax-for-delete useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "ref-uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"} domain = default-domain >> 09/01/2025 01:35:05.717 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 4de733ab-781c-4219-bcab-efd8adb97fdf object_type = security_group identifier_name = default-domain:k8s-default:k8s-default-default-sg url = http://10.0.0.50:8082/security-group/4de733ab-781c-4219-bcab-efd8adb97fdf operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": "4de733ab-781c-4219-bcab-efd8adb97fdf", "fq_name": ["default-domain", "k8s-default", "k8s-default-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "f3c16917-f766-498e-97d3-fb337c137bf5", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "56ebcf8e-11c5-49be-ac3c-78d905400c9a", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "230825e3-fa38-4f06-834a-0961de22c8e0", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "f7f355b5-0b47-4281-b975-82b09b907dd2", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 5613512272088154649, "uuid_lslong": 13595223614054563807}, "enable": true, "created": "2025-09-01T01:34:55.649202", "last_modified": "2025-09-01T01:34:55.724068", "description": "Default security group", "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:05.780 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-ea757534-9276-4848-bf9e-208ea86cff25', 'type': 'security_group', 'uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf'} 09/01/2025 01:35:05.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:05.789 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-list/0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "f3c16917-f766-498e-97d3-fb337c137bf5", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "56ebcf8e-11c5-49be-ac3c-78d905400c9a", "direction": null}]}, "access_control_list_hash": -7063133602176086075}} domain = default-domain >> 09/01/2025 01:35:05.796 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-10ea4a51-8f7f-43da-bee8-bb494de9a14b', 'type': 'access_control_list', 'uuid': '0a0f9403-fbe0-4cf4-8dd8-61b6703c9a68'} 09/01/2025 01:35:05.800 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e9c77d30-5ca4-4541-9585-66a4ccc4f9e1 object_type = access_control_list identifier_name = default-domain:k8s-default:k8s-default-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-list/e9c77d30-5ca4-4541-9585-66a4ccc4f9e1 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "e9c77d30-5ca4-4541-9585-66a4ccc4f9e1", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "230825e3-fa38-4f06-834a-0961de22c8e0", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "f7f355b5-0b47-4281-b975-82b09b907dd2", "direction": null}]}, "access_control_list_hash": 4875755405764241123}} domain = default-domain >> 09/01/2025 01:35:05.817 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-71caae77-7989-4f26-b852-4d8f72616c2e', 'type': 'access_control_list', 'uuid': 'e9c77d30-5ca4-4541-9585-66a4ccc4f9e1'} 09/01/2025 01:35:05.823 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 4de733ab-781c-4219-bcab-efd8adb97fdf object_type = security_group identifier_name = default-domain:k8s-default:k8s-default-default-sg url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "4de733ab-781c-4219-bcab-efd8adb97fdf", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-default"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "default"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "k8s-default-default-sg"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:05.830 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-99a26f15-1cc0-4bef-9a6e-db5b3355427e', 'type': 'security_group', 'uuid': '4de733ab-781c-4219-bcab-efd8adb97fdf'} 09/01/2025 01:35:05.837 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:05.837 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:05.890 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:05.891 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:05.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:05.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:05.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:05.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:05.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:05.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:05.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:05.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:05.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:05.951 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:05.952 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:05.952 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:05.952 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:05.952 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:05.953 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:05.954 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:05.954 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:05.954 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:05.959 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-default-ingress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-default-ingress 09/01/2025 01:35:05.963 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-default-ingress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-default-ingress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=default"], "tag_ids": [], "any": null}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-default-ingress"}} domain = default-domain >> 09/01/2025 01:35:06.026 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-default-ingress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-default-ingress', 'endpoint_1': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842754], 'tags': ['namespace=default'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-default-ingress'], 'id_perms': {'created': '2025-09-01T01:35:05.975858', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:05.975858', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13323639037654214571, 'uuid_mslong': 10651579802644989837}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=default'], 'uuid': 'faff8883-3e6f-4ea0-9683-30f13b91cd63'}], 'uuid': '93d20365-6161-478d-b8e7-131cf2e827ab'}, 'oper': 'CREATE', 'request-id': 'req-2fc07c57-fbcd-40c9-a93e-fd2c7c97d1fb', 'type': 'firewall_rule', 'uuid': '93d20365-6161-478d-b8e7-131cf2e827ab'} 09/01/2025 01:35:06.026 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:06.027 7fbb3cf35400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.027 7fbb3cf35400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-default-ingress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-default-ingress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=default"], "tag_ids": [16842754], "any": null}fq_name : ["default-policy-management", "k8s-Namespace-default-ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10651579802644989837, "uuid_lslong": 13323639037654214571}, "enable": true, "created": "2025-09-01T01:35:05.975858", "last_modified": "2025-09-01T01:35:05.975858", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=default"], "uuid": "faff8883-3e6f-4ea0-9683-30f13b91cd63"}]uuid : "93d20365-6161-478d-b8e7-131cf2e827ab" } deleted = False >> 09/01/2025 01:35:06.027 7fbb3cf35400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.031 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:06.031 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:06.031 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:06.031 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:06.031 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:06.080 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:06.080 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:06.081 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:06.085 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:06.085 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:06.085 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:06.085 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:06.086 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:06.092 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "93d20365-6161-478d-b8e7-131cf2e827ab", "ref-fq-name": ["default-policy-management", "k8s-Namespace-default-ingress"], "operation": "ADD", "attr": {"sequence": "00003.0"}} domain = default-domain >> 09/01/2025 01:35:06.111 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-2a936ac1-be50-4966-854f-efc81508be55', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:06.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:06.117 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:06.120 7fbb3cf35360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.120 7fbb3cf35360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:06.105969", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:06.120 7fbb3cf35360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:06.169 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:06.170 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:06.227 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:06.228 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:06.229 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:06.230 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:06.231 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:06.232 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:06.233 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:06.234 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:06.235 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:06.240 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-default-egress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-default-egress 09/01/2025 01:35:06.243 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-default-egress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-default-egress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=default"], "tag_ids": [], "any": null}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-default-egress"}} domain = default-domain >> 09/01/2025 01:35:06.319 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-default-egress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-default-egress', 'endpoint_1': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842754], 'tags': ['namespace=default'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-default-egress'], 'id_perms': {'created': '2025-09-01T01:35:06.261460', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:06.261460', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12047502222840008587, 'uuid_mslong': 17769216293693834069}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=default'], 'uuid': 'faff8883-3e6f-4ea0-9683-30f13b91cd63'}], 'uuid': 'f698f012-e2b4-4355-a731-5370f71b8f8b'}, 'oper': 'CREATE', 'request-id': 'req-31f11afa-0ead-4690-9c99-719bee03805c', 'type': 'firewall_rule', 'uuid': 'f698f012-e2b4-4355-a731-5370f71b8f8b'} 09/01/2025 01:35:06.320 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:06.321 7fbb3cf357c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.321 7fbb3cf357c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-default-egress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-default-egress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=default"], "tag_ids": [16842754], "any": null}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}fq_name : ["default-policy-management", "k8s-Namespace-default-egress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17769216293693834069, "uuid_lslong": 12047502222840008587}, "enable": true, "created": "2025-09-01T01:35:06.261460", "last_modified": "2025-09-01T01:35:06.261460", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=default"], "uuid": "faff8883-3e6f-4ea0-9683-30f13b91cd63"}]uuid : "f698f012-e2b4-4355-a731-5370f71b8f8b" } deleted = False >> 09/01/2025 01:35:06.321 7fbb3cf357c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:06.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:06.366 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:06.366 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:06.366 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:06.371 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:06.371 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:06.371 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:06.375 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:06.375 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:06.375 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:06.375 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:06.375 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:06.427 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b", "ref-fq-name": ["default-policy-management", "k8s-Namespace-default-egress"], "operation": "ADD", "attr": {"sequence": "00004.0"}} domain = default-domain >> 09/01/2025 01:35:06.454 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-f7649ee7-e03c-4af4-94d7-7b3cecade25a', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:06.457 7fbb3cf35360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.457 7fbb3cf35360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:06.449365", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:06.457 7fbb3cf35360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.458 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['k8s-svc=kube-dns'] not found in DB: Unknown id: tag k8s-svc=kube-dns 09/01/2025 01:35:06.458 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = k8s-svc=kube-dns url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["k8s-svc=kube-dns"], "tag_type_name": "k8s-svc", "tag_value": "kube-dns", "name": "k8s-svc=kube-dns", "display_name": "k8s-svc=kube-dns", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:06.490 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['k8s-svc=kube-dns'], 'obj_dict': {'display_name': 'k8s-svc=kube-dns', 'fq_name': ['k8s-svc=kube-dns'], 'id_perms': {'created': '2025-09-01T01:35:06.479406', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:06.479406', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11013834266846936573, 'uuid_mslong': 13924478071625763768}}, 'name': 'k8s-svc=kube-dns', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01030001', 'tag_type_name': 'k8s-svc', 'tag_type_refs': [{'to': ['k8s-svc'], 'uuid': 'd3e5d6f7-aece-4dc3-a032-9fe1561b040e'}], 'tag_value': 'kube-dns', 'uuid': 'c13daf07-f5f8-4bb8-98d8-ffe9726fbdfd'}, 'oper': 'CREATE', 'request-id': 'req-615b3138-4664-4587-a225-9d3958bd9152', 'type': 'tag', 'uuid': 'c13daf07-f5f8-4bb8-98d8-ffe9726fbdfd'} 09/01/2025 01:35:06.491 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:06.491 7fbb3cf357c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.491 7fbb3cf357c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = k8s-svc=kube-dns elements = { display_name : "k8s-svc=kube-dns"fq_name : ["k8s-svc=kube-dns"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 13924478071625763768, "uuid_lslong": 11013834266846936573}, "enable": true, "created": "2025-09-01T01:35:06.479406", "last_modified": "2025-09-01T01:35:06.479406", "description": null, "user_visible": true, "creator": null}name : "k8s-svc=kube-dns"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01030001"tag_type_name : "k8s-svc"tag_type_refs : [{"uuid": "d3e5d6f7-aece-4dc3-a032-9fe1561b040e", "to": ["k8s-svc"]}]tag_value : "kube-dns"uuid : "c13daf07-f5f8-4bb8-98d8-ffe9726fbdfd" } deleted = False >> 09/01/2025 01:35:06.491 7fbb3cf357c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.555 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:06.556 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:06.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:06.558 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:06.559 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:06.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:06.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:06.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:06.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:06.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:06.571 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:06.580 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:06.597 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-bd0c6ce6-1fc9-4d54-b986-551ec0ac81d7', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:06.600 7fbb3cf35a40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.600 7fbb3cf35a40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:06.589937", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:06.600 7fbb3cf35a40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:06.601 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:06.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:06.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:06.604 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:06.604 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:06.604 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:06.604 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:06.655 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:06.656 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:06.657 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:06.711 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliances in resource service-appliance-set: 'service_appliances' 09/01/2025 01:35:06.711 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource service-appliance-set: 'tag_refs' 09/01/2025 01:35:06.711 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource service-appliance-set: 'service_template_back_refs' 09/01/2025 01:35:06.712 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource service-appliance-set: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:06.712 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource service-appliance-set: 'loadbalancer_back_refs' 09/01/2025 01:35:06.721 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'] not found in DB: Unknown id: virtual_machine_interface default-domain:k8s-kube-system:k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 09/01/2025 01:35:06.756 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 38db5091-0c68-44f6-920d-1abb83a40be5 not found in DB: Unknown id: 38db5091-0c68-44f6-920d-1abb83a40be5 09/01/2025 01:35:06.757 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 38db5091-0c68-44f6-920d-1abb83a40be5 object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 url = http://10.0.0.50:8082/virtual-machine-interfaces operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-machine-interface": {"uuid": "38db5091-0c68-44f6-920d-1abb83a40be5", "fq_name": ["default-domain", "k8s-kube-system", "k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "parent_type": "project", "port_security_enabled": true, "virtual_machine_interface_device_owner": "K8S:LOADBALANCER", "display_name": "kube-system__kube-dns", "security_group_refs": [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"]}], "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:06.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:06.828 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'obj_dict': {'display_name': 'kube-system__kube-dns', 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'id_perms': {'created': '2025-09-01T01:35:06.764506', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:06.764506', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10524097296904489957, 'uuid_mslong': 4096956869932238070}}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'port_security_enabled': True, 'security_group_refs': [{'to': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg']}], 'uuid': '38db5091-0c68-44f6-920d-1abb83a40be5', 'virtual_machine_interface_device_owner': 'K8S:LOADBALANCER', 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:38:db:50:91:0c']}, 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'}]}, 'oper': 'CREATE', 'request-id': 'req-5af837c2-d2d1-4da8-9099-29707c80c392', 'type': 'virtual_machine_interface', 'uuid': '38db5091-0c68-44f6-920d-1abb83a40be5'} 09/01/2025 01:35:06.829 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVMITable:ContrailConfigTrace] map 09/01/2025 01:35:06.829 7fbb3cf35360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.829 7fbb3cf35360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 elements = { display_name : "kube-system__kube-dns"fq_name : ["default-domain", "k8s-kube-system", "k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 4096956869932238070, "uuid_lslong": 10524097296904489957}, "enable": true, "created": "2025-09-01T01:35:06.764506", "last_modified": "2025-09-01T01:35:06.764506", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truesecurity_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"]}]uuid : "38db5091-0c68-44f6-920d-1abb83a40be5"virtual_machine_interface_device_owner : "K8S:LOADBALANCER"virtual_machine_interface_mac_addresses : {"mac_address": ["02:38:db:50:91:0c"]}virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}] } deleted = False >> 09/01/2025 01:35:06.829 7fbb3cf35360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.837 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 38db5091-0c68-44f6-920d-1abb83a40be5 object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:06.855 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'oper': 'UPDATE', 'request-id': 'req-5af837c2-d2d1-4da8-9099-29707c80c392', 'type': 'virtual_machine_interface', 'uuid': '38db5091-0c68-44f6-920d-1abb83a40be5'} 09/01/2025 01:35:06.869 7fbb3cf35c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.870 7fbb3cf35c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 elements = { display_name : "kube-system__kube-dns"fq_name : ["default-domain", "k8s-kube-system", "k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 4096956869932238070, "uuid_lslong": 10524097296904489957}, "enable": true, "created": "2025-09-01T01:35:06.764506", "last_modified": "2025-09-01T01:35:06.844252", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network", "k8s-default-service-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "a099888b-9d11-4587-8b98-4d1673309c9e"}]security_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "attr": null, "uuid": "e0fa3e04-8145-47f6-9b69-718272132295"}]uuid : "38db5091-0c68-44f6-920d-1abb83a40be5"virtual_machine_interface_device_owner : "K8S:LOADBALANCER"virtual_machine_interface_mac_addresses : {"mac_address": ["02:38:db:50:91:0c"]}virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "attr": null, "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}] } deleted = False >> 09/01/2025 01:35:06.870 7fbb3cf35c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-machine-interface: 'virtual_machine_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:06.872 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:06.873 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:06.923 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2'] not found in DB: Unknown id: instance_ip kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2 09/01/2025 01:35:06.924 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 62fa6a29-784f-494c-91c4-4ca8a0b13fc2 not found in DB: Unknown id: 62fa6a29-784f-494c-91c4-4ca8a0b13fc2 09/01/2025 01:35:06.925 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 62fa6a29-784f-494c-91c4-4ca8a0b13fc2 object_type = instance_ip identifier_name = kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2 url = http://10.0.0.50:8082/instance-ips operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"instance-ip": {"uuid": "62fa6a29-784f-494c-91c4-4ca8a0b13fc2", "fq_name": ["kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2"], "instance_ip_address": "10.105.226.84", "subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400", "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kube-dns", "virtual_network_refs": [{"to": ["default-domain", "k8s-default", "k8s-default-service-network"], "uuid": "a6c41605-29b5-4568-a40b-17e3f103c269"}], "virtual_machine_interface_refs": [{"to": ["default-domain", "k8s-kube-system", "k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "uuid": "38db5091-0c68-44f6-920d-1abb83a40be5"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:06.935 7fbb3ceb7400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: AddrMgmt: alloc IIP 10.105.226.84 for vn=['default-domain', 'k8s-default', 'k8s-default-service-network'], tenant=default-project, askip=10.105.226.84 09/01/2025 01:35:06.951 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2'], 'obj_dict': {'display_name': 'kube-dns', 'fq_name': ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2'], 'id_perms': {'created': '2025-09-01T01:35:06.936398', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:06.936398', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10503604518069026754, 'uuid_mslong': 7132129686239267148}}, 'instance_ip_address': '10.105.226.84', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'subnet_uuid': 'cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400', 'uuid': '62fa6a29-784f-494c-91c4-4ca8a0b13fc2', 'virtual_machine_interface_refs': [{'to': ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'uuid': '38db5091-0c68-44f6-920d-1abb83a40be5'}], 'virtual_network_refs': [{'to': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'}]}, 'oper': 'CREATE', 'request-id': 'req-f5541a6a-8df6-4267-8dfe-656f78dc6123', 'type': 'instance_ip', 'uuid': '62fa6a29-784f-494c-91c4-4ca8a0b13fc2'} 09/01/2025 01:35:06.963 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:06.963 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:06.964 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:06.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'] not found in DB: Unknown id: loadbalancer default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 09/01/2025 01:35:06.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 0aaa8118-7dd7-4cf9-8d00-c56d41939884 not found in DB: Unknown id: 0aaa8118-7dd7-4cf9-8d00-c56d41939884 09/01/2025 01:35:06.975 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0aaa8118-7dd7-4cf9-8d00-c56d41939884 object_type = loadbalancer identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 url = http://10.0.0.50:8082/loadbalancers operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer": {"uuid": "0aaa8118-7dd7-4cf9-8d00-c56d41939884", "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "parent_type": "project", "loadbalancer_properties": {"status": null, "provisioning_status": "ACTIVE", "operating_status": "ONLINE", "vip_subnet_id": null, "vip_address": "10.105.226.84", "admin_state": true}, "loadbalancer_provider": "native", "display_name": "kube-system__kube-dns", "service_appliance_set_refs": [{"to": ["default-global-system-config", "native"], "uuid": "ce9684c7-029d-4e16-ad7f-6b14554d8775"}], "virtual_machine_interface_refs": [{"to": ["default-domain", "k8s-kube-system", "k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "uuid": "38db5091-0c68-44f6-920d-1abb83a40be5"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:07.035 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'obj_dict': {'display_name': 'kube-system__kube-dns', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'id_perms': {'created': '2025-09-01T01:35:06.982232', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:06.982232', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10160337832390137988, 'uuid_mslong': 768568628610551033}}, 'loadbalancer_properties': {'admin_state': True, 'operating_status': 'ONLINE', 'provisioning_status': 'ACTIVE', 'status': None, 'vip_address': '10.105.226.84', 'vip_subnet_id': None}, 'loadbalancer_provider': 'native', 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'service_appliance_set_refs': [{'to': ['default-global-system-config', 'native'], 'uuid': 'ce9684c7-029d-4e16-ad7f-6b14554d8775'}], 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884', 'virtual_machine_interface_refs': [{'to': ['default-domain', 'k8s-kube-system', 'k8s__Service__kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'uuid': '38db5091-0c68-44f6-920d-1abb83a40be5'}]}, 'oper': 'CREATE', 'request-id': 'req-e3172508-51f9-47f1-8974-4188a938672d', 'type': 'loadbalancer', 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884'} 09/01/2025 01:35:07.048 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.057 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.064 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-6ed18d80-3769-462c-a82c-86b3504280a7', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.067 7fbb3cf35cc0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.067 7fbb3cf35cc0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.060967", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.067 7fbb3cf35cc0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.068 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.069 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.070 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.071 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.076 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.076 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.076 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.076 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.076 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.077 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.077 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.077 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.078 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.083 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.084 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0aaa8118-7dd7-4cf9-8d00-c56d41939884 object_type = loadbalancer identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884 url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "0aaa8118-7dd7-4cf9-8d00-c56d41939884", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Service"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-system"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-system"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "kube-dns"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:07.099 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'oper': 'UPDATE', 'request-id': 'req-11cd59a6-02d3-4968-a81c-bbf3f6b2bba3', 'type': 'loadbalancer', 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884'} 09/01/2025 01:35:07.108 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.112 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.119 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-e5683e35-5c34-4c9a-8220-141ab3216dd0', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.121 7fbb3cf35360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.122 7fbb3cf35360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.115759", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.122 7fbb3cf35360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.123 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.123 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.123 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.123 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.123 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.124 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.125 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.126 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.130 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2', 'e2e22085-46d7-4e9f-9530-11e688ef29d7'] not found in DB: Unknown id: floating_ip kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2:e2e22085-46d7-4e9f-9530-11e688ef29d7 09/01/2025 01:35:07.133 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID e2e22085-46d7-4e9f-9530-11e688ef29d7 not found in DB: Unknown id: e2e22085-46d7-4e9f-9530-11e688ef29d7 09/01/2025 01:35:07.133 7fbb3d0e3900 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e2e22085-46d7-4e9f-9530-11e688ef29d7 object_type = floating_ip identifier_name = kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2:e2e22085-46d7-4e9f-9530-11e688ef29d7 url = http://10.0.0.50:8082/floating-ips operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-svc-monitor remote_ip = 10.0.0.50 body = {"floating-ip": {"uuid": "e2e22085-46d7-4e9f-9530-11e688ef29d7", "fq_name": ["kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2", "e2e22085-46d7-4e9f-9530-11e688ef29d7"], "parent_type": "instance-ip", "floating_ip_address": "10.105.226.84", "floating_ip_traffic_direction": "ingress", "project_refs": [{"to": ["default-domain", "k8s-kube-system"], "uuid": "a5dc0037-9655-44b1-8af6-f013fea66ee0"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "e2e22085-46d7-4e9f-9530-11e688ef29d7"}} domain = default-domain >> 09/01/2025 01:35:07.146 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2', 'e2e22085-46d7-4e9f-9530-11e688ef29d7'], 'obj_dict': {'display_name': 'e2e22085-46d7-4e9f-9530-11e688ef29d7', 'floating_ip_address': '10.105.226.84', 'floating_ip_traffic_direction': 'ingress', 'fq_name': ['kube-dns__62fa6a29-784f-494c-91c4-4ca8a0b13fc2', 'e2e22085-46d7-4e9f-9530-11e688ef29d7'], 'id_perms': {'created': '2025-09-01T01:35:07.137763', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.137763', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10750111992370899415, 'uuid_mslong': 16348665354099576479}}, 'parent_type': 'instance-ip', 'parent_uuid': '62fa6a29-784f-494c-91c4-4ca8a0b13fc2', 'perms2': {'global_access': 0, 'owner': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'owner_access': 7, 'share': []}, 'project_refs': [{'to': ['default-domain', 'k8s-kube-system'], 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'}], 'uuid': 'e2e22085-46d7-4e9f-9530-11e688ef29d7'}, 'oper': 'CREATE', 'request-id': 'req-73079181-47b0-4dec-98fa-025964e5bba1', 'type': 'floating_ip', 'uuid': 'e2e22085-46d7-4e9f-9530-11e688ef29d7'} 09/01/2025 01:35:07.174 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_refs in resource loadbalancer: 'service_instance_refs' 09/01/2025 01:35:07.174 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer: 'tag_refs' 09/01/2025 01:35:07.174 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource loadbalancer: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:07.178 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'] not found in DB: Unknown id: loadbalancer_listener default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df 09/01/2025 01:35:07.181 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID a8fccf8f-1536-430b-9b8a-f74dd27122df not found in DB: Unknown id: a8fccf8f-1536-430b-9b8a-f74dd27122df 09/01/2025 01:35:07.181 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a8fccf8f-1536-430b-9b8a-f74dd27122df object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df url = http://10.0.0.50:8082/loadbalancer-listeners operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-listener": {"uuid": "a8fccf8f-1536-430b-9b8a-f74dd27122df", "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df"], "parent_type": "project", "loadbalancer_listener_properties": {"protocol": "UDP", "protocol_port": 53, "admin_state": true, "connection_limit": null, "default_tls_container": null, "sni_containers": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df", "loadbalancer_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "uuid": "0aaa8118-7dd7-4cf9-8d00-c56d41939884"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:07.195 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'id_perms': {'created': '2025-09-01T01:35:07.186572', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.186572', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11208042536280072927, 'uuid_mslong': 12176835705946129163}}, 'loadbalancer_listener_properties': {'admin_state': True, 'connection_limit': None, 'default_tls_container': None, 'protocol': 'UDP', 'protocol_port': 53, 'sni_containers': []}, 'loadbalancer_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884'}], 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'a8fccf8f-1536-430b-9b8a-f74dd27122df'}, 'oper': 'CREATE', 'request-id': 'req-f8ccf411-b268-4e0d-ad61-4749ce522a22', 'type': 'loadbalancer_listener', 'uuid': 'a8fccf8f-1536-430b-9b8a-f74dd27122df'} 09/01/2025 01:35:07.205 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a8fccf8f-1536-430b-9b8a-f74dd27122df object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "a8fccf8f-1536-430b-9b8a-f74dd27122df", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "portName", "value": "dns"}, "position": "portName"}]} domain = default-domain >> 09/01/2025 01:35:07.213 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'oper': 'UPDATE', 'request-id': 'req-e040d7a9-7962-4c7e-89e1-24d1f5ba7333', 'type': 'loadbalancer_listener', 'uuid': 'a8fccf8f-1536-430b-9b8a-f74dd27122df'} 09/01/2025 01:35:07.260 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.265 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.276 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-0fe3f21a-58d0-42f3-967c-fd6c676916a3', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.281 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.281 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.268394", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.281 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.284 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.285 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.286 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.287 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.291 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.292 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.296 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_refs in resource loadbalancer-listener: 'loadbalancer_refs' 09/01/2025 01:35:07.296 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.297 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.345 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'] not found in DB: Unknown id: loadbalancer_pool default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df 09/01/2025 01:35:07.349 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = loadbalancer_pool identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df url = http://10.0.0.50:8082/loadbalancer-pools operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-pool": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df"], "parent_type": "project", "loadbalancer_pool_properties": {"status": null, "status_description": null, "admin_state": true, "protocol": "UDP", "loadbalancer_method": null, "subnet_id": null, "session_persistence": null, "persistence_cookie_name": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "loadbalancer_listener_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df"], "uuid": "a8fccf8f-1536-430b-9b8a-f74dd27122df"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df"}} domain = default-domain >> 09/01/2025 01:35:07.365 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'id_perms': {'created': '2025-09-01T01:35:07.357069', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.357069', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12369833172442127356, 'uuid_mslong': 328866569655238908}}, 'loadbalancer_listener_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-UDP-53-a8fccf8f-1536-430b-9b8a-f74dd27122df'], 'uuid': 'a8fccf8f-1536-430b-9b8a-f74dd27122df'}], 'loadbalancer_pool_properties': {'admin_state': True, 'loadbalancer_method': None, 'persistence_cookie_name': None, 'protocol': 'UDP', 'session_persistence': None, 'status': None, 'status_description': None, 'subnet_id': None}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '04905e67-16cd-40fc-abaa-79bb6e419bfc'}, 'oper': 'CREATE', 'request-id': 'req-c991a278-f364-4a90-ac3a-364e8750245e', 'type': 'loadbalancer_pool', 'uuid': '04905e67-16cd-40fc-abaa-79bb6e419bfc'} 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.374 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.378 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource instance-ip: 'virtual_network_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource instance-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.383 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.384 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.385 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.388 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource floating-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.388 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource floating-ip: 'tag_refs' 09/01/2025 01:35:07.388 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource floating-ip: 'customer_attachment_back_refs' 09/01/2025 01:35:07.393 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-c7c7a965-24fc-40d4-995d-1c21b4ccb98f', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.400 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.400 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.389364", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.400 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.401 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.404 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.404 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.404 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.404 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_refs in resource loadbalancer: 'service_instance_refs' 09/01/2025 01:35:07.450 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer: 'tag_refs' 09/01/2025 01:35:07.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource loadbalancer: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:07.454 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'] not found in DB: Unknown id: loadbalancer_listener default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3 09/01/2025 01:35:07.459 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID 096a7ee3-a48c-4713-949f-f75b8435fbb3 not found in DB: Unknown id: 096a7ee3-a48c-4713-949f-f75b8435fbb3 09/01/2025 01:35:07.459 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 096a7ee3-a48c-4713-949f-f75b8435fbb3 object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3 url = http://10.0.0.50:8082/loadbalancer-listeners operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-listener": {"uuid": "096a7ee3-a48c-4713-949f-f75b8435fbb3", "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3"], "parent_type": "project", "loadbalancer_listener_properties": {"protocol": "TCP", "protocol_port": 53, "admin_state": true, "connection_limit": null, "default_tls_container": null, "sni_containers": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3", "loadbalancer_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "uuid": "0aaa8118-7dd7-4cf9-8d00-c56d41939884"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:07.472 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'id_perms': {'created': '2025-09-01T01:35:07.463282', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.463282', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10709550411342543795, 'uuid_mslong': 678494210056013587}}, 'loadbalancer_listener_properties': {'admin_state': True, 'connection_limit': None, 'default_tls_container': None, 'protocol': 'TCP', 'protocol_port': 53, 'sni_containers': []}, 'loadbalancer_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884'}], 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '096a7ee3-a48c-4713-949f-f75b8435fbb3'}, 'oper': 'CREATE', 'request-id': 'req-689cc893-4473-47ec-b007-f31fa198879b', 'type': 'loadbalancer_listener', 'uuid': '096a7ee3-a48c-4713-949f-f75b8435fbb3'} 09/01/2025 01:35:07.482 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 096a7ee3-a48c-4713-949f-f75b8435fbb3 object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3 url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "096a7ee3-a48c-4713-949f-f75b8435fbb3", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "portName", "value": "dns-tcp"}, "position": "portName"}]} domain = default-domain >> 09/01/2025 01:35:07.490 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'oper': 'UPDATE', 'request-id': 'req-672a2903-2517-4396-bb13-c6d3b693205c', 'type': 'loadbalancer_listener', 'uuid': '096a7ee3-a48c-4713-949f-f75b8435fbb3'} 09/01/2025 01:35:07.540 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.545 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.554 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-6acf1880-92cd-41ec-82c2-dc00cee45367', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.561 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.561 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.548343", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.561 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.573 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_refs in resource loadbalancer-listener: 'loadbalancer_refs' 09/01/2025 01:35:07.573 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.573 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.577 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'] not found in DB: Unknown id: loadbalancer_pool default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3 09/01/2025 01:35:07.579 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = loadbalancer_pool identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3 url = http://10.0.0.50:8082/loadbalancer-pools operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-pool": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3"], "parent_type": "project", "loadbalancer_pool_properties": {"status": null, "status_description": null, "admin_state": true, "protocol": "TCP", "loadbalancer_method": null, "subnet_id": null, "session_persistence": null, "persistence_cookie_name": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "loadbalancer_listener_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3"], "uuid": "096a7ee3-a48c-4713-949f-f75b8435fbb3"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3"}} domain = default-domain >> 09/01/2025 01:35:07.600 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'id_perms': {'created': '2025-09-01T01:35:07.591632', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.591632', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12997013705438630066, 'uuid_mslong': 5282566568977188615}}, 'loadbalancer_listener_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-53-096a7ee3-a48c-4713-949f-f75b8435fbb3'], 'uuid': '096a7ee3-a48c-4713-949f-f75b8435fbb3'}], 'loadbalancer_pool_properties': {'admin_state': True, 'loadbalancer_method': None, 'persistence_cookie_name': None, 'protocol': 'TCP', 'session_persistence': None, 'status': None, 'status_description': None, 'subnet_id': None}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '494f724e-663d-4707-b45e-ab1a9d8d90b2'}, 'oper': 'CREATE', 'request-id': 'req-665d527a-7f15-4ca7-8d71-91ad818ac7a0', 'type': 'loadbalancer_pool', 'uuid': '494f724e-663d-4707-b45e-ab1a9d8d90b2'} 09/01/2025 01:35:07.607 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.608 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.611 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.611 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource instance-ip: 'virtual_network_refs' 09/01/2025 01:35:07.611 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource instance-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.612 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.615 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource floating-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.615 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource floating-ip: 'tag_refs' 09/01/2025 01:35:07.615 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource floating-ip: 'customer_attachment_back_refs' 09/01/2025 01:35:07.649 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.653 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.662 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.663 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.665 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.666 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.667 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.668 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-0d33716b-4d2c-4b04-80a7-9505fc4c84b1', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.672 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.672 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.656972", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.672 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.673 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_refs in resource loadbalancer: 'service_instance_refs' 09/01/2025 01:35:07.673 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer: 'tag_refs' 09/01/2025 01:35:07.673 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource loadbalancer: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:07.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'] not found in DB: Unknown id: loadbalancer_listener default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63 09/01/2025 01:35:07.684 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: UUID d29b75ec-b179-4d0b-9e50-01234a890d63 not found in DB: Unknown id: d29b75ec-b179-4d0b-9e50-01234a890d63 09/01/2025 01:35:07.684 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d29b75ec-b179-4d0b-9e50-01234a890d63 object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63 url = http://10.0.0.50:8082/loadbalancer-listeners operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-listener": {"uuid": "d29b75ec-b179-4d0b-9e50-01234a890d63", "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63"], "parent_type": "project", "loadbalancer_listener_properties": {"protocol": "TCP", "protocol_port": 9153, "admin_state": true, "connection_limit": null, "default_tls_container": null, "sni_containers": []}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63", "loadbalancer_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884"], "uuid": "0aaa8118-7dd7-4cf9-8d00-c56d41939884"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:07.699 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'id_perms': {'created': '2025-09-01T01:35:07.689058', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.689058', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11407619107215445347, 'uuid_mslong': 15175853028805266699}}, 'loadbalancer_listener_properties': {'admin_state': True, 'connection_limit': None, 'default_tls_container': None, 'protocol': 'TCP', 'protocol_port': 9153, 'sni_containers': []}, 'loadbalancer_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884'], 'uuid': '0aaa8118-7dd7-4cf9-8d00-c56d41939884'}], 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'd29b75ec-b179-4d0b-9e50-01234a890d63'}, 'oper': 'CREATE', 'request-id': 'req-8b9baf47-a7af-4cb8-beb3-49a05b8164db', 'type': 'loadbalancer_listener', 'uuid': 'd29b75ec-b179-4d0b-9e50-01234a890d63'} 09/01/2025 01:35:07.740 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d29b75ec-b179-4d0b-9e50-01234a890d63 object_type = loadbalancer_listener identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63 url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "d29b75ec-b179-4d0b-9e50-01234a890d63", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "portName", "value": "metrics"}, "position": "portName"}]} domain = default-domain >> 09/01/2025 01:35:07.749 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'oper': 'UPDATE', 'request-id': 'req-9f31a74c-b813-4e82-ba2d-4d9c64acf448', 'type': 'loadbalancer_listener', 'uuid': 'd29b75ec-b179-4d0b-9e50-01234a890d63'} 09/01/2025 01:35:07.754 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:07.762 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:07.770 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-e2309acc-fa5d-4797-adc5-34a2c98e1b29', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:07.779 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.780 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:07.765123", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:07.780 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:07.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:07.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:07.782 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:07.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:07.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:07.830 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.831 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_refs in resource loadbalancer-listener: 'loadbalancer_refs' 09/01/2025 01:35:07.836 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource loadbalancer-listener: 'tag_refs' 09/01/2025 01:35:07.836 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource loadbalancer-listener: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.840 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'] not found in DB: Unknown id: loadbalancer_pool default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63 09/01/2025 01:35:07.843 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = loadbalancer_pool identifier_name = default-domain:k8s-kube-system:kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63 url = http://10.0.0.50:8082/loadbalancer-pools operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"loadbalancer-pool": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63"], "parent_type": "project", "loadbalancer_pool_properties": {"status": null, "status_description": null, "admin_state": true, "protocol": "TCP", "loadbalancer_method": null, "subnet_id": null, "session_persistence": null, "persistence_cookie_name": null}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "loadbalancer_listener_refs": [{"to": ["default-domain", "k8s-kube-system", "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63"], "uuid": "d29b75ec-b179-4d0b-9e50-01234a890d63"}], "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63"}} domain = default-domain >> 09/01/2025 01:35:07.867 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'obj_dict': {'display_name': 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63', 'fq_name': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'id_perms': {'created': '2025-09-01T01:35:07.857798', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.857798', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12107355557576793757, 'uuid_mslong': 12602419387071219849}}, 'loadbalancer_listener_refs': [{'to': ['default-domain', 'k8s-kube-system', 'kube-dns__0aaa8118-7dd7-4cf9-8d00-c56d41939884-TCP-9153-d29b75ec-b179-4d0b-9e50-01234a890d63'], 'uuid': 'd29b75ec-b179-4d0b-9e50-01234a890d63'}], 'loadbalancer_pool_properties': {'admin_state': True, 'loadbalancer_method': None, 'persistence_cookie_name': None, 'protocol': 'TCP', 'session_persistence': None, 'status': None, 'status_description': None, 'subnet_id': None}, 'parent_type': 'project', 'parent_uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': 'aee4c9a9-7ce0-4889-a805-f7bb6ab75e9d'}, 'oper': 'CREATE', 'request-id': 'req-43656287-d4ff-46d5-a655-da77ddad12ea', 'type': 'loadbalancer_pool', 'uuid': 'aee4c9a9-7ce0-4889-a805-f7bb6ab75e9d'} 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ips in resource instance-ip: 'floating_ips' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.878 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.882 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource instance-ip: 'parent_type' 09/01/2025 01:35:07.882 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource instance-ip: 'virtual_network_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource instance-ip: 'network_ipam_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource instance-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_refs in resource instance-ip: 'physical_router_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_refs in resource instance-ip: 'virtual_router_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_refs in resource instance-ip: 'logical_interface_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_refs in resource instance-ip: 'flow_node_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource instance-ip: 'tag_refs' 09/01/2025 01:35:07.883 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource instance-ip: 'service_instance_back_refs' 09/01/2025 01:35:07.911 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name=kube-node-lease'] not found in DB: Unknown id: tag kubernetes.io__slash__metadata.name=kube-node-lease 09/01/2025 01:35:07.911 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = kubernetes.io__slash__metadata.name=kube-node-lease url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name=kube-node-lease"], "tag_type_name": "kubernetes.io__slash__metadata.name", "tag_value": "kube-node-lease", "name": "kubernetes.io__slash__metadata.name=kube-node-lease", "display_name": "kubernetes.io__slash__metadata.name=kube-node-lease", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:07.930 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name=kube-node-lease'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name=kube-node-lease', 'fq_name': ['kubernetes.io__slash__metadata.name=kube-node-lease'], 'id_perms': {'created': '2025-09-01T01:35:07.921452', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:07.921452', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13374079485892415267, 'uuid_mslong': 15718588142150699747}}, 'name': 'kubernetes.io__slash__metadata.name=kube-node-lease', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01020002', 'tag_type_name': 'kubernetes.io__slash__metadata.name', 'tag_type_refs': [{'to': ['kubernetes.io__slash__metadata.name'], 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}], 'tag_value': 'kube-node-lease', 'uuid': 'da23a4a2-778f-4ae3-b99a-466efa74cf23'}, 'oper': 'CREATE', 'request-id': 'req-96744b7d-1c4d-455b-a918-7b19300b1a83', 'type': 'tag', 'uuid': 'da23a4a2-778f-4ae3-b99a-466efa74cf23'} 09/01/2025 01:35:07.931 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:07.931 7fbb3cf35c20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.932 7fbb3cf35c20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = kubernetes.io__slash__metadata.name=kube-node-lease elements = { display_name : "kubernetes.io__slash__metadata.name=kube-node-lease"fq_name : ["kubernetes.io__slash__metadata.name=kube-node-lease"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15718588142150699747, "uuid_lslong": 13374079485892415267}, "enable": true, "created": "2025-09-01T01:35:07.921452", "last_modified": "2025-09-01T01:35:07.921452", "description": null, "user_visible": true, "creator": null}name : "kubernetes.io__slash__metadata.name=kube-node-lease"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01020002"tag_type_name : "kubernetes.io__slash__metadata.name"tag_type_refs : [{"uuid": "c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6", "to": ["kubernetes.io__slash__metadata.name"]}]tag_value : "kube-node-lease"uuid : "da23a4a2-778f-4ae3-b99a-466efa74cf23" } deleted = False >> 09/01/2025 01:35:07.932 7fbb3cf35c20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:07.939 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource floating-ip: 'virtual_machine_interface_refs' 09/01/2025 01:35:07.939 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource floating-ip: 'tag_refs' 09/01/2025 01:35:07.939 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource floating-ip: 'customer_attachment_back_refs' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:07.942 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:07.943 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:07.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:07.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:07.946 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:07.947 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:07.948 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:07.949 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:07.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:07.950 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:07.994 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace=kube-node-lease'] not found in DB: Unknown id: tag namespace=kube-node-lease 09/01/2025 01:35:07.995 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = namespace=kube-node-lease url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["namespace=kube-node-lease"], "tag_type_name": "namespace", "tag_value": "kube-node-lease", "name": "namespace=kube-node-lease", "display_name": "namespace=kube-node-lease", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:08.033 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace=kube-node-lease'], 'obj_dict': {'display_name': 'namespace=kube-node-lease', 'fq_name': ['namespace=kube-node-lease'], 'id_perms': {'created': '2025-09-01T01:35:08.016756', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.016756', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13078349777989594095, 'uuid_mslong': 11288639297156038718}}, 'name': 'namespace=kube-node-lease', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01010003', 'tag_type_name': 'namespace', 'tag_type_refs': [{'to': ['namespace'], 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}], 'tag_value': 'kube-node-lease', 'uuid': '9ca94da1-8880-403e-b57f-a1d4bd60ebef'}, 'oper': 'CREATE', 'request-id': 'req-56619a5e-043b-439e-a45a-1ae7eb3863d1', 'type': 'tag', 'uuid': '9ca94da1-8880-403e-b57f-a1d4bd60ebef'} 09/01/2025 01:35:08.034 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:08.034 7fbb3cf5ab80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.034 7fbb3cf5ab80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = namespace=kube-node-lease elements = { display_name : "namespace=kube-node-lease"fq_name : ["namespace=kube-node-lease"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11288639297156038718, "uuid_lslong": 13078349777989594095}, "enable": true, "created": "2025-09-01T01:35:08.016756", "last_modified": "2025-09-01T01:35:08.016756", "description": null, "user_visible": true, "creator": null}name : "namespace=kube-node-lease"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01010003"tag_type_name : "namespace"tag_type_refs : [{"uuid": "884e5052-64fc-44a5-a883-82d91352fa8e", "to": ["namespace"]}]tag_value : "kube-node-lease"uuid : "9ca94da1-8880-403e-b57f-a1d4bd60ebef" } deleted = False >> 09/01/2025 01:35:08.034 7fbb3cf5ab80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:08.054 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:08.055 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:08.056 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:08.057 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:08.058 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:08.059 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:08.060 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:08.061 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:08.062 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:08.069 7fbb3cf5ab80 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:35:08.071 7fbb3cf5ab80 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:35:08.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease'] not found in DB: Unknown id: project default-domain:k8s-kube-node-lease 09/01/2025 01:35:08.111 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = project identifier_name = default-domain:k8s-kube-node-lease url = http://10.0.0.50:8082/projects operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"project": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease"], "parent_type": "domain", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-node-lease"}} domain = default-domain >> 09/01/2025 01:35:08.128 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease'], 'obj_dict': {'display_name': 'k8s-kube-node-lease', 'fq_name': ['default-domain', 'k8s-kube-node-lease'], 'id_perms': {'created': '2025-09-01T01:35:08.117860', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.117860', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13243892127741515929, 'uuid_mslong': 10235971928760011485}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899'}, 'oper': 'CREATE', 'request-id': 'req-67755f30-ee01-481f-82ec-6bdf3ec8ed6d', 'type': 'project', 'uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899'} 09/01/2025 01:35:08.129 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:35:08.130 7fbb3cf35360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.130 7fbb3cf35360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-node-lease elements = { display_name : "k8s-kube-node-lease"fq_name : ["default-domain", "k8s-kube-node-lease"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10235971928760011485, "uuid_lslong": 13243892127741515929}, "enable": true, "created": "2025-09-01T01:35:08.117860", "last_modified": "2025-09-01T01:35:08.117860", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "8e0d7a3c-989c-4add-b7cb-c1b895a01899" } deleted = False >> 09/01/2025 01:35:08.130 7fbb3cf35360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.132 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'default-application-policy-set'] not found in DB: Unknown id: application_policy_set default-domain:k8s-kube-node-lease:default-application-policy-set 09/01/2025 01:35:08.134 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-domain:k8s-kube-node-lease:default-application-policy-set url = http://127.0.0.1/application-policy-sets operation = post body = {"application-policy-set": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "default-application-policy-set"], "parent_type": "project", "all_applications": true, "display_name": "default-application-policy-set", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:08.147 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'display_name': 'default-application-policy-set', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:35:08.141506', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.141506', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11599686833657261955, 'uuid_mslong': 6557066945879952406}}, 'parent_type': 'project', 'parent_uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '5aff61a5-7da1-4c16-a0fa-5db7a9eaaf83'}, 'oper': 'CREATE', 'request-id': 'req-67755f30-ee01-481f-82ec-6bdf3ec8ed6d', 'type': 'application_policy_set', 'uuid': '5aff61a5-7da1-4c16-a0fa-5db7a9eaaf83'} 09/01/2025 01:35:08.147 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:35:08.148 7fbb3cf35d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.148 7fbb3cf35d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-node-lease:default-application-policy-set elements = { all_applications : truedisplay_name : "default-application-policy-set"fq_name : ["default-domain", "k8s-kube-node-lease", "default-application-policy-set"]id_perms : {"permissions": {"owner": "contrail-api", "owner_access": 7, "group": "admin", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6557066945879952406, "uuid_lslong": 11599686833657261955}, "enable": true, "created": "2025-09-01T01:35:08.141506", "last_modified": "2025-09-01T01:35:08.141506", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "8e0d7a3c-989c-4add-b7cb-c1b895a01899"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "5aff61a5-7da1-4c16-a0fa-5db7a9eaaf83" } deleted = False >> 09/01/2025 01:35:08.148 7fbb3cf35d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.152 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 8e0d7a3c-989c-4add-b7cb-c1b895a01899 object_type = project identifier_name = default-domain:k8s-kube-node-lease url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:08.166 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease'], 'oper': 'UPDATE', 'request-id': 'req-67755f30-ee01-481f-82ec-6bdf3ec8ed6d', 'type': 'project', 'uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899'} 09/01/2025 01:35:08.170 7fbb3cf35f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.170 7fbb3cf35f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-node-lease elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-node-lease", "default-application-policy-set"], "attr": null, "uuid": "5aff61a5-7da1-4c16-a0fa-5db7a9eaaf83"}]display_name : "k8s-kube-node-lease"fq_name : ["default-domain", "k8s-kube-node-lease"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10235971928760011485, "uuid_lslong": 13243892127741515929}, "enable": true, "created": "2025-09-01T01:35:08.117860", "last_modified": "2025-09-01T01:35:08.156650", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "8e0d7a3c-989c-4add-b7cb-c1b895a01899" } deleted = False >> 09/01/2025 01:35:08.170 7fbb3cf35f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.176 7fbb3d0e3ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'default'] not found in DB: Unknown id: security_group default-domain:k8s-kube-node-lease:default 09/01/2025 01:35:08.176 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-node-lease:default url = http://cn-jenkins-deploy-platform-juju-k8s-1454-1:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-api remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "default"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "1bf977a9-083f-4f52-adc5-794fce9a63e5", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-node-lease:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "2393b834-6985-4b82-8b4e-39ac9c7dc3ce", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-node-lease:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "1c72fa75-47db-4cb2-adee-4ab2a119da26", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "ae9fe320-160f-4589-a736-93a29502bcb9", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "default"}} domain = default-domain >> 09/01/2025 01:35:08.243 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default'], 'obj_dict': {'display_name': 'default', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default'], 'id_perms': {'created': '2025-09-01T01:35:08.189203', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:08.189203', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10355378182601732884, 'uuid_mslong': 3631824606228794343}}, 'parent_type': 'project', 'parent_uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'perms2': {'global_access': 0, 'owner': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '1bf977a9-083f-4f52-adc5-794fce9a63e5', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-node-lease:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '2393b834-6985-4b82-8b4e-39ac9c7dc3ce', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-node-lease:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '1c72fa75-47db-4cb2-adee-4ab2a119da26', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'ae9fe320-160f-4589-a736-93a29502bcb9', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000008, 'uuid': '3266d53c-b4ca-4be7-8fb5-b19788ce7314'}, 'oper': 'CREATE', 'request-id': 'req-adcf92cc-3838-4d8c-b02f-1e4b7123e458', 'type': 'security_group', 'uuid': '3266d53c-b4ca-4be7-8fb5-b19788ce7314'} 09/01/2025 01:35:08.252 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 8e0d7a3c-989c-4add-b7cb-c1b895a01899 object_type = project identifier_name = default-domain:k8s-kube-node-lease url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-node-lease"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-node-lease"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "kube-node-lease"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": "e0db9dca-7fc8-47d9-89a4-88f04377b390"}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "isolated", "value": "False"}, "position": "isolated"}]} domain = default-domain >> 09/01/2025 01:35:08.258 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease'], 'oper': 'UPDATE', 'request-id': 'req-c8dd4699-30f4-4302-8658-d1c9c7e961c2', 'type': 'project', 'uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899'} 09/01/2025 01:35:08.262 7fbb3cf35f40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.263 7fbb3cf35f40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-node-lease elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "isolated", "value": "False"}, {"key": "k8s_uuid", "value": "e0db9dca-7fc8-47d9-89a4-88f04377b390"}, {"key": "kind", "value": "Namespace"}, {"key": "name", "value": "kube-node-lease"}, {"key": "namespace", "value": "kube-node-lease"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-node-lease"}]}application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-node-lease", "default-application-policy-set"], "attr": null, "uuid": "5aff61a5-7da1-4c16-a0fa-5db7a9eaaf83"}]display_name : "k8s-kube-node-lease"fq_name : ["default-domain", "k8s-kube-node-lease"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 10235971928760011485, "uuid_lslong": 13243892127741515929}, "enable": true, "created": "2025-09-01T01:35:08.117860", "last_modified": "2025-09-01T01:35:08.253690", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "8e0d7a3c-989c-4add-b7cb-c1b895a01899" } deleted = False >> 09/01/2025 01:35:08.263 7fbb3cf35f40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.264 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'default', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-node-lease:default:ingress-access-control-list 09/01/2025 01:35:08.266 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-node-lease:default:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "default", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000008", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "1bf977a9-083f-4f52-adc5-794fce9a63e5", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000008", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "2393b834-6985-4b82-8b4e-39ac9c7dc3ce", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:08.310 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:08.311 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:08.312 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:08.318 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:08.319 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:08.320 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:08.388 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-network/d650e01f-39d4-4eb1-91f8-ca54373fcba6 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:08.497 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000008', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '1bf977a9-083f-4f52-adc5-794fce9a63e5'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000008', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '2393b834-6985-4b82-8b4e-39ac9c7dc3ce'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:08.443304', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.443304', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13779271877774155511, 'uuid_mslong': 1901039727771534284}}, 'parent_type': 'security-group', 'parent_uuid': '3266d53c-b4ca-4be7-8fb5-b19788ce7314', 'perms2': {'global_access': 0, 'owner': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'owner_access': 7, 'share': []}, 'uuid': '1a61d98f-6dba-43cc-bf39-cec462421ef7'}, 'oper': 'CREATE', 'request-id': 'req-627a7f7f-388f-416a-bef6-8db21c7f4970', 'type': 'access_control_list', 'uuid': '1a61d98f-6dba-43cc-bf39-cec462421ef7'} 09/01/2025 01:35:08.501 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-3bda41c6-5d8b-415a-9fd7-14913639ec04', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:08.516 7fbb3d30eea0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.516 7fbb3d30eea0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:08.490681", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:08.516 7fbb3d30eea0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.519 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'default', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-node-lease:default:egress-access-control-list 09/01/2025 01:35:08.540 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-node-lease:default:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "default", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "1c72fa75-47db-4cb2-adee-4ab2a119da26", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "ae9fe320-160f-4589-a736-93a29502bcb9", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:08.543 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:08.560 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-3bda41c6-5d8b-415a-9fd7-14913639ec04', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:08.601 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-network/a6c41605-29b5-4568-a40b-17e3f103c269 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:08.605 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '1c72fa75-47db-4cb2-adee-4ab2a119da26'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'ae9fe320-160f-4589-a736-93a29502bcb9'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'default', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:08.552827', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.552827', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12223284962926760408, 'uuid_mslong': 9767097960267923502}}, 'parent_type': 'security-group', 'parent_uuid': '3266d53c-b4ca-4be7-8fb5-b19788ce7314', 'perms2': {'global_access': 0, 'owner': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'owner_access': 7, 'share': []}, 'uuid': '878bb3d8-b836-402e-a9a1-d4e97b9f91d8'}, 'oper': 'CREATE', 'request-id': 'req-390b984d-1730-424d-bdf5-3edc2e41b892', 'type': 'access_control_list', 'uuid': '878bb3d8-b836-402e-a9a1-d4e97b9f91d8'} 09/01/2025 01:35:08.625 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-de605c4f-b2e0-41c2-9951-be9556a27795', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:08.629 7fbb3d30eb80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.629 7fbb3d30eb80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:08.619326", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:08.629 7fbb3d30eb80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:08.636 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:08.644 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-de605c4f-b2e0-41c2-9951-be9556a27795', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:08.648 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg'] not found in DB: Unknown id: security_group default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg 09/01/2025 01:35:08.657 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg url = http://10.0.0.50:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "k8s-kube-node-lease-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "01335bfd-0746-431c-918c-c51187711b8b", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "7dae5209-fb44-4db7-9366-8c4f5e9a37b1", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "27af981d-28c1-44b2-97f0-b4212fb494b7", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "824ed245-0ba2-44e1-9f82-f7c6fe54ba48", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-node-lease-default-sg"}} domain = default-domain >> 09/01/2025 01:35:08.728 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg'], 'obj_dict': {'display_name': 'k8s-kube-node-lease-default-sg', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg'], 'id_perms': {'created': '2025-09-01T01:35:08.671049', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:08.671049', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12277904507868437021, 'uuid_mslong': 17077533445816337081}}, 'parent_type': 'project', 'parent_uuid': '8e0d7a3c-989c-4add-b7cb-c1b895a01899', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '01335bfd-0746-431c-918c-c51187711b8b', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '7dae5209-fb44-4db7-9366-8c4f5e9a37b1', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '27af981d-28c1-44b2-97f0-b4212fb494b7', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '824ed245-0ba2-44e1-9f82-f7c6fe54ba48', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000009, 'uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d'}, 'oper': 'CREATE', 'request-id': 'req-ac7d6189-024e-4169-b43d-76c3a6d20aef', 'type': 'security_group', 'uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d'} 09/01/2025 01:35:08.746 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg:ingress-access-control-list 09/01/2025 01:35:08.749 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "k8s-kube-node-lease-default-sg", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "01335bfd-0746-431c-918c-c51187711b8b", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "7dae5209-fb44-4db7-9366-8c4f5e9a37b1", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:08.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:08.802 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = ecff9630-35bc-42b9-aa63-e11949aa5a1d object_type = security_group identifier_name = default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "ecff9630-35bc-42b9-aa63-e11949aa5a1d", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-node-lease"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-node-lease"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "k8s-kube-node-lease-default-sg"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:08.812 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '01335bfd-0746-431c-918c-c51187711b8b'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '7dae5209-fb44-4db7-9366-8c4f5e9a37b1'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:08.756795', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.756795', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11466293722866954989, 'uuid_mslong': 6670521070751402776}}, 'parent_type': 'security-group', 'parent_uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '5c927392-68e5-4b18-9f20-7563d1c6b2ed'}, 'oper': 'CREATE', 'request-id': 'req-f69b0936-4451-4dea-ad2e-7fc5f9534daf', 'type': 'access_control_list', 'uuid': '5c927392-68e5-4b18-9f20-7563d1c6b2ed'} 09/01/2025 01:35:08.830 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg:egress-access-control-list 09/01/2025 01:35:08.849 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-877457c7-d08c-4c98-99ac-53ff0411afdc', 'type': 'security_group', 'uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d'} 09/01/2025 01:35:08.851 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-node-lease:k8s-kube-node-lease-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-node-lease", "k8s-kube-node-lease-default-sg", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "27af981d-28c1-44b2-97f0-b4212fb494b7", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "824ed245-0ba2-44e1-9f82-f7c6fe54ba48", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:08.874 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: chown: ecff9630-35bc-42b9-aa63-e11949aa5a1d owner set to 8e0d7a3c-989c-4add-b7cb-c1b895a01899 09/01/2025 01:35:08.879 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-26beae75-376d-42fc-a43f-c3482dc143d1', 'type': 'security_group', 'uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d'} 09/01/2025 01:35:08.906 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:08.914 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:08.953 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '27af981d-28c1-44b2-97f0-b4212fb494b7'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '824ed245-0ba2-44e1-9f82-f7c6fe54ba48'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-node-lease', 'k8s-kube-node-lease-default-sg', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:08.901065', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:08.901065', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12329592412285012411, 'uuid_mslong': 51089021332179352}}, 'parent_type': 'security-group', 'parent_uuid': 'ecff9630-35bc-42b9-aa63-e11949aa5a1d', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '00b58131-b86d-4998-ab1b-82f9579989bb'}, 'oper': 'CREATE', 'request-id': 'req-3d943d03-6797-4d8a-bd47-f83455eca619', 'type': 'access_control_list', 'uuid': '00b58131-b86d-4998-ab1b-82f9579989bb'} 09/01/2025 01:35:08.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:08.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:08.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:08.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:08.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:08.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:08.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:09.034 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:09.035 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:09.036 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:09.037 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:09.038 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:09.039 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:09.042 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:09.043 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:09.044 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:09.045 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:09.045 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:09.045 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:09.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-node-lease-ingress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-node-lease-ingress 09/01/2025 01:35:09.053 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-node-lease-ingress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-node-lease"], "tag_ids": [], "any": null}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-node-lease-ingress"}} domain = default-domain >> 09/01/2025 01:35:09.127 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-node-lease-ingress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-node-lease-ingress', 'endpoint_1': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842755], 'tags': ['namespace=kube-node-lease'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-node-lease-ingress'], 'id_perms': {'created': '2025-09-01T01:35:09.068780', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.068780', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9791100812219705833, 'uuid_mslong': 17402610047194647621}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-node-lease'], 'uuid': '9ca94da1-8880-403e-b57f-a1d4bd60ebef'}], 'uuid': 'f1827da2-8bae-4c45-87e0-fa503064e9e9'}, 'oper': 'CREATE', 'request-id': 'req-bb6fa775-a0ca-48ca-8b64-5f578ab02999', 'type': 'firewall_rule', 'uuid': 'f1827da2-8bae-4c45-87e0-fa503064e9e9'} 09/01/2025 01:35:09.128 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:09.128 7fbb3cfbbc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.129 7fbb3cfbbc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-node-lease-ingress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-node-lease-ingress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-node-lease"], "tag_ids": [16842755], "any": null}fq_name : ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17402610047194647621, "uuid_lslong": 9791100812219705833}, "enable": true, "created": "2025-09-01T01:35:09.068780", "last_modified": "2025-09-01T01:35:09.068780", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-node-lease"], "uuid": "9ca94da1-8880-403e-b57f-a1d4bd60ebef"}]uuid : "f1827da2-8bae-4c45-87e0-fa503064e9e9" } deleted = False >> 09/01/2025 01:35:09.129 7fbb3cfbbc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.168 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:09.168 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:09.168 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:09.168 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:09.168 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:09.173 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:09.173 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:09.173 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:09.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:09.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:09.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:09.176 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:09.177 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:09.182 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "operation": "ADD", "attr": {"sequence": "00005.0"}} domain = default-domain >> 09/01/2025 01:35:09.187 7fbb3d0e3040 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:09.188 7fbb3d0e3040 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:09.188 7fbb3d0e3040 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:09.188 7fbb3d0e3040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:09.188 7fbb3d0e3040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:09.189 7fbb3d0e3040 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.204 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-e7805a5b-0948-4ef5-801c-53f81f0cf097', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:09.207 7fbb3d0e3180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.208 7fbb3d0e3180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:09.201218", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:09.208 7fbb3d0e3180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.209 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:09.209 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:09.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:09.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:09.261 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:09.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:09.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:09.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:09.262 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:09.270 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:09.271 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:09.272 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:09.273 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:09.274 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:09.275 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:09.276 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:09.277 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:09.322 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-node-lease-egress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-node-lease-egress 09/01/2025 01:35:09.325 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-node-lease-egress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-node-lease"], "tag_ids": [], "any": null}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-node-lease-egress"}} domain = default-domain >> 09/01/2025 01:35:09.384 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-node-lease-egress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-node-lease-egress', 'endpoint_1': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842755], 'tags': ['namespace=kube-node-lease'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-node-lease-egress'], 'id_perms': {'created': '2025-09-01T01:35:09.334663', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.334663', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11673476119293380291, 'uuid_mslong': 16111642645617723340}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-node-lease'], 'uuid': '9ca94da1-8880-403e-b57f-a1d4bd60ebef'}], 'uuid': 'df980d9f-8014-47cc-a200-84ae887cc6c3'}, 'oper': 'CREATE', 'request-id': 'req-4ab8947c-05ac-452c-af37-fcdf76759d32', 'type': 'firewall_rule', 'uuid': 'df980d9f-8014-47cc-a200-84ae887cc6c3'} 09/01/2025 01:35:09.385 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:09.385 7fbb3cfbbc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.385 7fbb3cfbbc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-node-lease-egress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-node-lease-egress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-node-lease"], "tag_ids": [16842755], "any": null}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}fq_name : ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16111642645617723340, "uuid_lslong": 11673476119293380291}, "enable": true, "created": "2025-09-01T01:35:09.334663", "last_modified": "2025-09-01T01:35:09.334663", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-node-lease"], "uuid": "9ca94da1-8880-403e-b57f-a1d4bd60ebef"}]uuid : "df980d9f-8014-47cc-a200-84ae887cc6c3" } deleted = False >> 09/01/2025 01:35:09.385 7fbb3cfbbc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.391 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:09.391 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:09.391 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:09.391 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:09.391 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:09.402 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:09.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:09.403 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:09.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:09.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:09.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:09.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:09.451 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:09.461 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "operation": "ADD", "attr": {"sequence": "00006.0"}} domain = default-domain >> 09/01/2025 01:35:09.502 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-9e6d47a6-1950-4eec-81b8-2065014e081b', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:09.505 7fbb3d0e3180 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.505 7fbb3d0e3180 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "attr": {"sequence": "00006.0"}, "uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:09.489700", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:09.505 7fbb3d0e3180 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.506 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name=kube-public'] not found in DB: Unknown id: tag kubernetes.io__slash__metadata.name=kube-public 09/01/2025 01:35:09.506 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = kubernetes.io__slash__metadata.name=kube-public url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name=kube-public"], "tag_type_name": "kubernetes.io__slash__metadata.name", "tag_value": "kube-public", "name": "kubernetes.io__slash__metadata.name=kube-public", "display_name": "kubernetes.io__slash__metadata.name=kube-public", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:09.536 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name=kube-public'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name=kube-public', 'fq_name': ['kubernetes.io__slash__metadata.name=kube-public'], 'id_perms': {'created': '2025-09-01T01:35:09.525668', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.525668', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12857215920637343278, 'uuid_mslong': 6157200344163633711}}, 'name': 'kubernetes.io__slash__metadata.name=kube-public', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01020003', 'tag_type_name': 'kubernetes.io__slash__metadata.name', 'tag_type_refs': [{'to': ['kubernetes.io__slash__metadata.name'], 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}], 'tag_value': 'kube-public', 'uuid': '5572c517-3cc9-4a2f-b26e-01c26615f62e'}, 'oper': 'CREATE', 'request-id': 'req-4378319e-3e3b-4973-b6fa-04b78261c136', 'type': 'tag', 'uuid': '5572c517-3cc9-4a2f-b26e-01c26615f62e'} 09/01/2025 01:35:09.537 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:09.538 7fbb3cfbb720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.539 7fbb3cfbb720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = kubernetes.io__slash__metadata.name=kube-public elements = { display_name : "kubernetes.io__slash__metadata.name=kube-public"fq_name : ["kubernetes.io__slash__metadata.name=kube-public"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 6157200344163633711, "uuid_lslong": 12857215920637343278}, "enable": true, "created": "2025-09-01T01:35:09.525668", "last_modified": "2025-09-01T01:35:09.525668", "description": null, "user_visible": true, "creator": null}name : "kubernetes.io__slash__metadata.name=kube-public"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01020003"tag_type_name : "kubernetes.io__slash__metadata.name"tag_type_refs : [{"uuid": "c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6", "to": ["kubernetes.io__slash__metadata.name"]}]tag_value : "kube-public"uuid : "5572c517-3cc9-4a2f-b26e-01c26615f62e" } deleted = False >> 09/01/2025 01:35:09.539 7fbb3cfbb720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:09.580 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:09.581 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:09.582 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:09.583 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:09.584 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:09.584 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:09.585 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:09.586 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:09.587 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:09.588 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:09.589 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:09.594 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['namespace=kube-public'] not found in DB: Unknown id: tag namespace=kube-public 09/01/2025 01:35:09.595 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = namespace=kube-public url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["namespace=kube-public"], "tag_type_name": "namespace", "tag_value": "kube-public", "name": "namespace=kube-public", "display_name": "namespace=kube-public", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:09.619 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['namespace=kube-public'], 'obj_dict': {'display_name': 'namespace=kube-public', 'fq_name': ['namespace=kube-public'], 'id_perms': {'created': '2025-09-01T01:35:09.612231', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.612231', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11728912746587169791, 'uuid_mslong': 17558590392535762952}}, 'name': 'namespace=kube-public', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01010004', 'tag_type_name': 'namespace', 'tag_type_refs': [{'to': ['namespace'], 'uuid': '884e5052-64fc-44a5-a883-82d91352fa8e'}], 'tag_value': 'kube-public', 'uuid': 'f3aca4ee-bfae-4808-a2c5-780025e123ff'}, 'oper': 'CREATE', 'request-id': 'req-ca3d8c77-1039-413f-9d2b-b53c4a26afe7', 'type': 'tag', 'uuid': 'f3aca4ee-bfae-4808-a2c5-780025e123ff'} 09/01/2025 01:35:09.620 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:09.620 7fbb3cfbb5e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.620 7fbb3cfbb5e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = namespace=kube-public elements = { display_name : "namespace=kube-public"fq_name : ["namespace=kube-public"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 17558590392535762952, "uuid_lslong": 11728912746587169791}, "enable": true, "created": "2025-09-01T01:35:09.612231", "last_modified": "2025-09-01T01:35:09.612231", "description": null, "user_visible": true, "creator": null}name : "namespace=kube-public"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01010004"tag_type_name : "namespace"tag_type_refs : [{"uuid": "884e5052-64fc-44a5-a883-82d91352fa8e", "to": ["namespace"]}]tag_value : "kube-public"uuid : "f3aca4ee-bfae-4808-a2c5-780025e123ff" } deleted = False >> 09/01/2025 01:35:09.620 7fbb3cfbb5e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.674 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:09.675 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:09.676 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:09.677 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:09.678 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:09.679 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:09.680 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:09.681 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:09.682 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:09.683 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:09.689 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public'] not found in DB: Unknown id: project default-domain:k8s-kube-public 09/01/2025 01:35:09.689 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = project identifier_name = default-domain:k8s-kube-public url = http://10.0.0.50:8082/projects operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"project": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public"], "parent_type": "domain", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-public"}} domain = default-domain >> 09/01/2025 01:35:09.716 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public'], 'obj_dict': {'display_name': 'k8s-kube-public', 'fq_name': ['default-domain', 'k8s-kube-public'], 'id_perms': {'created': '2025-09-01T01:35:09.709338', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.709338', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12382982719713215992, 'uuid_mslong': 1059935213013714696}}, 'parent_type': 'domain', 'parent_uuid': 'ce056a4b-12ea-41a5-896d-393d506f2011', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8'}, 'oper': 'CREATE', 'request-id': 'req-69797a7b-9f8e-4b7f-a6af-3652d2ea5c0d', 'type': 'project', 'uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8'} 09/01/2025 01:35:09.716 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectProjectTable:ContrailConfigTrace] map 09/01/2025 01:35:09.716 7fbb3cfbb720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.717 7fbb3cfbb720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-public elements = { display_name : "k8s-kube-public"fq_name : ["default-domain", "k8s-kube-public"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1059935213013714696, "uuid_lslong": 12382982719713215992}, "enable": true, "created": "2025-09-01T01:35:09.709338", "last_modified": "2025-09-01T01:35:09.709338", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "0eb5a575-e09c-4708-abd9-312cfc4e85f8" } deleted = False >> 09/01/2025 01:35:09.717 7fbb3cfbb720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.719 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'default-application-policy-set'] not found in DB: Unknown id: application_policy_set default-domain:k8s-kube-public:default-application-policy-set 09/01/2025 01:35:09.725 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = application_policy_set identifier_name = default-domain:k8s-kube-public:default-application-policy-set url = http://127.0.0.1/application-policy-sets operation = post body = {"application-policy-set": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "default-application-policy-set"], "parent_type": "project", "all_applications": true, "display_name": "default-application-policy-set", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:09.749 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'default-application-policy-set'], 'obj_dict': {'all_applications': True, 'display_name': 'default-application-policy-set', 'fq_name': ['default-domain', 'k8s-kube-public', 'default-application-policy-set'], 'id_perms': {'created': '2025-09-01T01:35:09.744535', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.744535', 'permissions': {'group': 'admin', 'group_access': 7, 'other_access': 7, 'owner': 'contrail-api', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 9582523444762837509, 'uuid_mslong': 5637291694719647816}}, 'parent_type': 'project', 'parent_uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '4e3baeed-bf06-4048-84fb-f64d78f10205'}, 'oper': 'CREATE', 'request-id': 'req-69797a7b-9f8e-4b7f-a6af-3652d2ea5c0d', 'type': 'application_policy_set', 'uuid': '4e3baeed-bf06-4048-84fb-f64d78f10205'} 09/01/2025 01:35:09.750 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectApplicationPolicySetTable:ContrailConfigTrace] map 09/01/2025 01:35:09.750 7fbb3cfbbae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.753 7fbb3cfbbae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-public:default-application-policy-set elements = { all_applications : truedisplay_name : "default-application-policy-set"fq_name : ["default-domain", "k8s-kube-public", "default-application-policy-set"]id_perms : {"permissions": {"owner": "contrail-api", "owner_access": 7, "group": "admin", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 5637291694719647816, "uuid_lslong": 9582523444762837509}, "enable": true, "created": "2025-09-01T01:35:09.744535", "last_modified": "2025-09-01T01:35:09.744535", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "0eb5a575-e09c-4708-abd9-312cfc4e85f8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "4e3baeed-bf06-4048-84fb-f64d78f10205" } deleted = False >> 09/01/2025 01:35:09.753 7fbb3cfbbae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.755 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0eb5a575-e09c-4708-abd9-312cfc4e85f8 object_type = project identifier_name = default-domain:k8s-kube-public url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:09.773 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public'], 'oper': 'UPDATE', 'request-id': 'req-69797a7b-9f8e-4b7f-a6af-3652d2ea5c0d', 'type': 'project', 'uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8'} 09/01/2025 01:35:09.776 7fbb3cfbbae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.776 7fbb3cfbbae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-public elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-public", "default-application-policy-set"], "attr": null, "uuid": "4e3baeed-bf06-4048-84fb-f64d78f10205"}]display_name : "k8s-kube-public"fq_name : ["default-domain", "k8s-kube-public"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1059935213013714696, "uuid_lslong": 12382982719713215992}, "enable": true, "created": "2025-09-01T01:35:09.709338", "last_modified": "2025-09-01T01:35:09.765268", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "0eb5a575-e09c-4708-abd9-312cfc4e85f8" } deleted = False >> 09/01/2025 01:35:09.777 7fbb3cfbbae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:09.789 7fbb3d0e3ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'default'] not found in DB: Unknown id: security_group default-domain:k8s-kube-public:default 09/01/2025 01:35:09.790 7fbb3d0e3ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-public:default url = http://cn-jenkins-deploy-platform-juju-k8s-1454-1:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-api remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "default"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "89a68178-7737-4108-80a0-4986bfcd3106", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-public:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "9c3b6f70-ae91-47c8-9df3-67b2252ae2e5", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "default-domain:k8s-kube-public:default", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "67b96713-a84f-4494-b1c6-c2c2977ce9aa", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "84c84276-8154-458c-8198-d0d82912d3e2", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "default"}} domain = default-domain >> 09/01/2025 01:35:09.859 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'default'], 'obj_dict': {'display_name': 'default', 'fq_name': ['default-domain', 'k8s-kube-public', 'default'], 'id_perms': {'created': '2025-09-01T01:35:09.803936', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:09.803936', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10347208463162891061, 'uuid_mslong': 4265717819942062928}}, 'parent_type': 'project', 'parent_uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'perms2': {'global_access': 0, 'owner': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '89a68178-7737-4108-80a0-4986bfcd3106', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-public:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '9c3b6f70-ae91-47c8-9df3-67b2252ae2e5', 'src_addresses': [{'network_policy': None, 'security_group': 'default-domain:k8s-kube-public:default', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '67b96713-a84f-4494-b1c6-c2c2977ce9aa', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '84c84276-8154-458c-8198-d0d82912d3e2', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000010, 'uuid': '3b32dfc1-a9b3-4b50-8f98-ab467fbd5b35'}, 'oper': 'CREATE', 'request-id': 'req-5b85b15d-7427-4962-8c7d-ac5d9951d4dc', 'type': 'security_group', 'uuid': '3b32dfc1-a9b3-4b50-8f98-ab467fbd5b35'} 09/01/2025 01:35:09.887 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'default', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-public:default:ingress-access-control-list 09/01/2025 01:35:09.890 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-public:default:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "default", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000010", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "89a68178-7737-4108-80a0-4986bfcd3106", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": "8000010", "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "9c3b6f70-ae91-47c8-9df3-67b2252ae2e5", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:09.948 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 0eb5a575-e09c-4708-abd9-312cfc4e85f8 object_type = project identifier_name = default-domain:k8s-kube-public url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-public"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-public"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "kube-public"}, "position": "name"}, {"field": "annotations", "operation": "set", "value": {"key": "k8s_uuid", "value": "4422e8b3-382f-457d-8515-b4e0ab6cae7b"}, "position": "k8s_uuid"}, {"field": "annotations", "operation": "set", "value": {"key": "isolated", "value": "False"}, "position": "isolated"}]} domain = default-domain >> 09/01/2025 01:35:09.960 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'default', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000010', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '89a68178-7737-4108-80a0-4986bfcd3106'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': '8000010', 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '9c3b6f70-ae91-47c8-9df3-67b2252ae2e5'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-public', 'default', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:09.901575', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:09.901575', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10631142203170786979, 'uuid_mslong': 10153433766146494016}}, 'parent_type': 'security-group', 'parent_uuid': '3b32dfc1-a9b3-4b50-8f98-ab467fbd5b35', 'perms2': {'global_access': 0, 'owner': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'owner_access': 7, 'share': []}, 'uuid': '8ce83e37-119e-4e40-9389-6781f0a88ea3'}, 'oper': 'CREATE', 'request-id': 'req-b2df4bdd-510a-4cec-859a-569a47bede46', 'type': 'access_control_list', 'uuid': '8ce83e37-119e-4e40-9389-6781f0a88ea3'} 09/01/2025 01:35:09.972 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public'], 'oper': 'UPDATE', 'request-id': 'req-0d9c0810-b84e-4dc7-8de0-7749d2662d96', 'type': 'project', 'uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8'} 09/01/2025 01:35:09.985 7fbb3cfbbc20 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.985 7fbb3cfbbc20 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-public elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "isolated", "value": "False"}, {"key": "k8s_uuid", "value": "4422e8b3-382f-457d-8515-b4e0ab6cae7b"}, {"key": "kind", "value": "Namespace"}, {"key": "name", "value": "kube-public"}, {"key": "namespace", "value": "kube-public"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-public"}]}application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-public", "default-application-policy-set"], "attr": null, "uuid": "4e3baeed-bf06-4048-84fb-f64d78f10205"}]display_name : "k8s-kube-public"fq_name : ["default-domain", "k8s-kube-public"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 1059935213013714696, "uuid_lslong": 12382982719713215992}, "enable": true, "created": "2025-09-01T01:35:09.709338", "last_modified": "2025-09-01T01:35:09.961962", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "0eb5a575-e09c-4708-abd9-312cfc4e85f8" } deleted = False >> 09/01/2025 01:35:09.985 7fbb3cfbbc20 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:09.992 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'default', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-public:default:egress-access-control-list 09/01/2025 01:35:09.996 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:09.998 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:09.999 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:10.001 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:10.005 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-public:default:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "default", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "67b96713-a84f-4494-b1c6-c2c2977ce9aa", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "84c84276-8154-458c-8198-d0d82912d3e2", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:10.050 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:10.051 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:10.052 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:10.053 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:10.057 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-network/d650e01f-39d4-4eb1-91f8-ca54373fcba6 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:10.071 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-336387db-eb60-4c6f-b119-859f4528e9b5', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:10.075 7fbb3d1df7c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.077 7fbb3d1df7c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:10.066298", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:10.077 7fbb3d1df7c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.089 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:10.140 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'default', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '67b96713-a84f-4494-b1c6-c2c2977ce9aa'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '84c84276-8154-458c-8198-d0d82912d3e2'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-public', 'default', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:10.087967', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:10.087967', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13567071936738506524, 'uuid_mslong': 10770495866812713647}}, 'parent_type': 'security-group', 'parent_uuid': '3b32dfc1-a9b3-4b50-8f98-ab467fbd5b35', 'perms2': {'global_access': 0, 'owner': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'owner_access': 7, 'share': []}, 'uuid': '95787ced-242f-4aaf-bc47-ec0b8778371c'}, 'oper': 'CREATE', 'request-id': 'req-10689d71-6487-4a8e-8116-c61e7f4b0245', 'type': 'access_control_list', 'uuid': '95787ced-242f-4aaf-bc47-ec0b8778371c'} 09/01/2025 01:35:10.148 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-336387db-eb60-4c6f-b119-859f4528e9b5', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:10.195 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-network/a6c41605-29b5-4568-a40b-17e3f103c269 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:10.209 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-0e66adc2-242b-42f1-9769-0e79d035d3a3', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:10.212 7fbb3cfbbae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.212 7fbb3cfbbae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:10.201181", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:10.212 7fbb3cfbbae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.219 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:10.227 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-0e66adc2-242b-42f1-9769-0e79d035d3a3', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:10.245 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg'] not found in DB: Unknown id: security_group default-domain:k8s-kube-public:k8s-kube-public-default-sg 09/01/2025 01:35:10.248 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = security_group identifier_name = default-domain:k8s-kube-public:k8s-kube-public-default-sg url = http://10.0.0.50:8082/security-groups operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "k8s-kube-public-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "14e9c1ba-6529-4291-bf18-e7857b87aab3", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "1afc35ea-9414-4d93-8c2b-028e2d41e118", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "c2ea3148-20d1-4715-9670-bc1f2d678de9", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "84d3d5d3-f956-4e38-a9e1-e3f1b77f0521", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": "Default security group", "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-kube-public-default-sg"}} domain = default-domain >> 09/01/2025 01:35:10.324 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg'], 'obj_dict': {'display_name': 'k8s-kube-public-default-sg', 'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg'], 'id_perms': {'created': '2025-09-01T01:35:10.267151', 'creator': None, 'description': 'Default security group', 'enable': True, 'last_modified': '2025-09-01T01:35:10.267151', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12841608011934621753, 'uuid_mslong': 4609916113583950431}}, 'parent_type': 'project', 'parent_uuid': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'security_group_entries': {'policy_rule': [{'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '14e9c1ba-6529-4291-bf18-e7857b87aab3', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '1afc35ea-9414-4d93-8c2b-028e2d41e118', 'src_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv4', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': 'c2ea3148-20d1-4715-9670-bc1f2d678de9', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}, {'action_list': None, 'application': [], 'created': None, 'description': None, 'direction': '>', 'dst_addresses': [{'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}], 'dst_ports': [{'end_port': 65535, 'start_port': 0}], 'ethertype': 'IPv6', 'last_modified': None, 'protocol': 'any', 'rule_sequence': None, 'rule_uuid': '84d3d5d3-f956-4e38-a9e1-e3f1b77f0521', 'src_addresses': [{'network_policy': None, 'security_group': 'local', 'subnet': None, 'subnet_list': [], 'virtual_network': None}], 'src_ports': [{'end_port': 65535, 'start_port': 0}]}]}, 'security_group_id': 8000011, 'uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039'}, 'oper': 'CREATE', 'request-id': 'req-22391dd1-b0ec-43b0-bc2b-a3ce20ed6755', 'type': 'security_group', 'uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039'} 09/01/2025 01:35:10.328 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 3ff9b647-ea85-4a5f-b236-8e72f82c6039 object_type = security_group identifier_name = default-domain:k8s-kube-public:k8s-kube-public-default-sg url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "3ff9b647-ea85-4a5f-b236-8e72f82c6039", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-public"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-public"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "k8s-kube-public-default-sg"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:10.351 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-7f54334e-d06d-4159-923c-4e9e6e78bcd1', 'type': 'security_group', 'uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039'} 09/01/2025 01:35:10.353 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'ingress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-public:k8s-kube-public-default-sg:ingress-access-control-list 09/01/2025 01:35:10.359 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-public:k8s-kube-public-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "k8s-kube-public-default-sg", "ingress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "14e9c1ba-6529-4291-bf18-e7857b87aab3", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "1afc35ea-9414-4d93-8c2b-028e2d41e118", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "ingress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:10.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_NOTICE]: VncApiNotice: chown: 3ff9b647-ea85-4a5f-b236-8e72f82c6039 owner set to 0eb5a575-e09c-4708-abd9-312cfc4e85f8 09/01/2025 01:35:10.367 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-d20e9dc9-64db-45da-8c5e-3d79e8f31f69', 'type': 'security_group', 'uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039'} 09/01/2025 01:35:10.433 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:10.434 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:10.447 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'ingress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '14e9c1ba-6529-4291-bf18-e7857b87aab3'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '1afc35ea-9414-4d93-8c2b-028e2d41e118'}], 'dynamic': None}, 'display_name': 'ingress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'ingress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:10.386974', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:10.386974', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13390437058598049228, 'uuid_mslong': 5502608932187098816}}, 'parent_type': 'security-group', 'parent_uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039', 'perms2': {'global_access': 0, 'owner': 'None', 'owner_access': 7, 'share': []}, 'uuid': '4c5d31ab-74e7-4ec0-b9d4-638f28f695cc'}, 'oper': 'CREATE', 'request-id': 'req-e64d83b6-7531-49c9-ba60-e2c61eb819a2', 'type': 'access_control_list', 'uuid': '4c5d31ab-74e7-4ec0-b9d4-638f28f695cc'} 09/01/2025 01:35:10.466 7fbb3d1f6180 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'egress-access-control-list'] not found in DB: Unknown id: access_control_list default-domain:k8s-kube-public:k8s-kube-public-default-sg:egress-access-control-list 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:10.472 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:10.475 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = access_control_list identifier_name = default-domain:k8s-kube-public:k8s-kube-public-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-lists operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": null, "fq_name": ["default-domain", "k8s-kube-public", "k8s-kube-public-default-sg", "egress-access-control-list"], "parent_type": "security-group", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "c2ea3148-20d1-4715-9670-bc1f2d678de9", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "84d3d5d3-f956-4e38-a9e1-e3f1b77f0521", "direction": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "egress-access-control-list"}} domain = default-domain >> 09/01/2025 01:35:10.555 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'egress-access-control-list'], 'obj_dict': {'access_control_list_entries': {'acl_rule': [{'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '0.0.0.0', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv4', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': 'c2ea3148-20d1-4715-9670-bc1f2d678de9'}, {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': None, 'match_condition': {'dst_address': {'network_policy': None, 'security_group': None, 'subnet': {'ip_prefix': '::', 'ip_prefix_len': 0}, 'subnet_list': [], 'virtual_network': None}, 'dst_port': {'end_port': 65535, 'start_port': 0}, 'ethertype': 'IPv6', 'protocol': 'any', 'src_address': {'network_policy': None, 'security_group': None, 'subnet': None, 'subnet_list': [], 'virtual_network': None}, 'src_port': {'end_port': 65535, 'start_port': 0}}, 'rule_uuid': '84d3d5d3-f956-4e38-a9e1-e3f1b77f0521'}], 'dynamic': None}, 'display_name': 'egress-access-control-list', 'fq_name': ['default-domain', 'k8s-kube-public', 'k8s-kube-public-default-sg', 'egress-access-control-list'], 'id_perms': {'created': '2025-09-01T01:35:10.498098', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:10.498098', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13038786667139761078, 'uuid_mslong': 14716788366151010158}}, 'parent_type': 'security-group', 'parent_uuid': '3ff9b647-ea85-4a5f-b236-8e72f82c6039', 'perms2': {'global_access': 0, 'owner': '0eb5a575-e09c-4708-abd9-312cfc4e85f8', 'owner_access': 7, 'share': []}, 'uuid': 'cc3c890b-cf08-476e-b4f3-13642cf703b6'}, 'oper': 'CREATE', 'request-id': 'req-70d96687-406b-47d9-919b-c0e5eb49f2de', 'type': 'access_control_list', 'uuid': 'cc3c890b-cf08-476e-b4f3-13642cf703b6'} 09/01/2025 01:35:10.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:10.557 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:10.560 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:10.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:10.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:10.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:10.561 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:10.562 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:10.563 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:10.564 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:10.565 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:10.566 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:10.567 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:10.568 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:10.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:10.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:10.569 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:10.620 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:10.622 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:10.623 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:10.623 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:10.623 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:10.623 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:10.623 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:10.672 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-public-ingress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-public-ingress 09/01/2025 01:35:10.679 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-public-ingress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-public"], "tag_ids": [], "any": null}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-public-ingress"}} domain = default-domain >> 09/01/2025 01:35:10.769 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-public-ingress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-public-ingress', 'endpoint_1': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842756], 'tags': ['namespace=kube-public'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-public-ingress'], 'id_perms': {'created': '2025-09-01T01:35:10.711118', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:10.711118', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10607826179290505792, 'uuid_mslong': 13188655992088249721}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-public'], 'uuid': 'f3aca4ee-bfae-4808-a2c5-780025e123ff'}], 'uuid': 'b70784bf-913f-4579-9336-91b517b32240'}, 'oper': 'CREATE', 'request-id': 'req-49b6e189-d5b9-4351-9c22-d694d7476369', 'type': 'firewall_rule', 'uuid': 'b70784bf-913f-4579-9336-91b517b32240'} 09/01/2025 01:35:10.770 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:10.770 7fbb3d1df7c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.771 7fbb3d1df7c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-public-ingress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-public-ingress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-public"], "tag_ids": [16842756], "any": null}fq_name : ["default-policy-management", "k8s-Namespace-kube-public-ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 13188655992088249721, "uuid_lslong": 10607826179290505792}, "enable": true, "created": "2025-09-01T01:35:10.711118", "last_modified": "2025-09-01T01:35:10.711118", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-public"], "uuid": "f3aca4ee-bfae-4808-a2c5-780025e123ff"}]uuid : "b70784bf-913f-4579-9336-91b517b32240" } deleted = False >> 09/01/2025 01:35:10.771 7fbb3d1df7c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.777 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:10.777 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:10.778 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:10.778 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:10.778 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:10.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:10.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:10.783 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:10.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:10.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:10.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:10.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:10.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:10.835 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:10.842 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "b70784bf-913f-4579-9336-91b517b32240", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "operation": "ADD", "attr": {"sequence": "00007.0"}} domain = default-domain >> 09/01/2025 01:35:10.890 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-d3290e7e-7c81-444e-8025-4446899997ac', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:10.892 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:10.892 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:10.895 7fbb3cfbb720 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.895 7fbb3cfbb720 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "attr": {"sequence": "00007.0"}, "uuid": "b70784bf-913f-4579-9336-91b517b32240"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "attr": {"sequence": "00006.0"}, "uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:10.881139", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:10.895 7fbb3cfbb720 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:10.944 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:10.945 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:11.000 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:11.000 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:11.000 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:11.002 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:11.002 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:11.002 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:11.003 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:11.004 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:11.005 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:11.006 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:11.007 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:11.008 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:11.009 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:11.010 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:11.011 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:11.011 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:11.018 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-public-egress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-public-egress 09/01/2025 01:35:11.022 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-public-egress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-public-egress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-public"], "tag_ids": [], "any": null}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-public-egress"}} domain = default-domain >> 09/01/2025 01:35:11.085 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-public-egress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-public-egress', 'endpoint_1': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842756], 'tags': ['namespace=kube-public'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-public-egress'], 'id_perms': {'created': '2025-09-01T01:35:11.034553', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:11.034553', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12687764789551225607, 'uuid_mslong': 7321813596295873554}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-public'], 'uuid': 'f3aca4ee-bfae-4808-a2c5-780025e123ff'}], 'uuid': '659c4eac-56d7-4412-b013-feda7ae8db07'}, 'oper': 'CREATE', 'request-id': 'req-d04bb5ab-e8db-4ce0-9dcb-da64fd7f9641', 'type': 'firewall_rule', 'uuid': '659c4eac-56d7-4412-b013-feda7ae8db07'} 09/01/2025 01:35:11.086 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:11.086 7fbb3d1df7c0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.086 7fbb3d1df7c0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-public-egress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-public-egress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-public"], "tag_ids": [16842756], "any": null}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}fq_name : ["default-policy-management", "k8s-Namespace-kube-public-egress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 7321813596295873554, "uuid_lslong": 12687764789551225607}, "enable": true, "created": "2025-09-01T01:35:11.034553", "last_modified": "2025-09-01T01:35:11.034553", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-public"], "uuid": "f3aca4ee-bfae-4808-a2c5-780025e123ff"}]uuid : "659c4eac-56d7-4412-b013-feda7ae8db07" } deleted = False >> 09/01/2025 01:35:11.086 7fbb3d1df7c0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.129 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:11.129 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:11.129 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:11.129 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:11.129 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:11.134 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:11.134 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:11.134 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:11.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:11.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:11.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:11.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:11.140 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:11.196 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "659c4eac-56d7-4412-b013-feda7ae8db07", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-public-egress"], "operation": "ADD", "attr": {"sequence": "00008.0"}} domain = default-domain >> 09/01/2025 01:35:11.245 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-c9cfc777-6c6e-4025-a0f0-2046f5cc631e', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:11.249 7fbb3cfbbd60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.249 7fbb3cfbbd60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-egress"], "attr": {"sequence": "00008.0"}, "uuid": "659c4eac-56d7-4412-b013-feda7ae8db07"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "attr": {"sequence": "00007.0"}, "uuid": "b70784bf-913f-4579-9336-91b517b32240"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "attr": {"sequence": "00006.0"}, "uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:11.232518", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:11.249 7fbb3cfbbd60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.259 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['kubernetes.io__slash__metadata.name=kube-system'] not found in DB: Unknown id: tag kubernetes.io__slash__metadata.name=kube-system 09/01/2025 01:35:11.259 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = tag identifier_name = kubernetes.io__slash__metadata.name=kube-system url = http://10.0.0.50:8082/tags operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"tag": {"uuid": null, "fq_name": ["kubernetes.io__slash__metadata.name=kube-system"], "tag_type_name": "kubernetes.io__slash__metadata.name", "tag_value": "kube-system", "name": "kubernetes.io__slash__metadata.name=kube-system", "display_name": "kubernetes.io__slash__metadata.name=kube-system", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}}} domain = default-domain >> 09/01/2025 01:35:11.297 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['kubernetes.io__slash__metadata.name=kube-system'], 'obj_dict': {'display_name': 'kubernetes.io__slash__metadata.name=kube-system', 'fq_name': ['kubernetes.io__slash__metadata.name=kube-system'], 'id_perms': {'created': '2025-09-01T01:35:11.284227', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:11.284227', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 13811631599716858853, 'uuid_mslong': 8548833652210091014}}, 'name': 'kubernetes.io__slash__metadata.name=kube-system', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'tag_id': '0x01020004', 'tag_type_name': 'kubernetes.io__slash__metadata.name', 'tag_type_refs': [{'to': ['kubernetes.io__slash__metadata.name'], 'uuid': 'c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6'}], 'tag_value': 'kube-system', 'uuid': '76a38ee9-bbe3-4406-bfac-c5c345aefbe5'}, 'oper': 'CREATE', 'request-id': 'req-4b8f40b3-4cd2-4cca-8a4d-9bb8bf2211f4', 'type': 'tag', 'uuid': '76a38ee9-bbe3-4406-bfac-c5c345aefbe5'} 09/01/2025 01:35:11.297 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectTagTable:ContrailConfigTrace] map 09/01/2025 01:35:11.299 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.299 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = kubernetes.io__slash__metadata.name=kube-system elements = { display_name : "kubernetes.io__slash__metadata.name=kube-system"fq_name : ["kubernetes.io__slash__metadata.name=kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 8548833652210091014, "uuid_lslong": 13811631599716858853}, "enable": true, "created": "2025-09-01T01:35:11.284227", "last_modified": "2025-09-01T01:35:11.284227", "description": null, "user_visible": true, "creator": null}name : "kubernetes.io__slash__metadata.name=kube-system"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}tag_id : "0x01020004"tag_type_name : "kubernetes.io__slash__metadata.name"tag_type_refs : [{"uuid": "c4c51d7c-32fa-45a0-bfe9-26e1923cb4f6", "to": ["kubernetes.io__slash__metadata.name"]}]tag_value : "kube-system"uuid : "76a38ee9-bbe3-4406-bfac-c5c345aefbe5" } deleted = False >> 09/01/2025 01:35:11.299 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:11.354 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:11.355 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:11.356 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:11.357 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:11.358 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:11.359 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:11.360 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:11.361 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:11.362 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:11.363 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:11.364 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:11.365 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:11.366 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:11.423 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:11.424 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:11.425 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:11.426 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:11.427 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:11.428 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:11.429 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:11.430 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:11.485 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Error in pre_project_read extension: AttributeError: 'ResourceApiDriver' object has no attribute '_keystone_sync_on_demand' 09/01/2025 01:35:11.491 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a5dc0037-9655-44b1-8af6-f013fea66ee0 object_type = project identifier_name = default-domain:k8s-kube-system url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:11.502 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system'], 'oper': 'UPDATE', 'request-id': 'req-f03f9d16-99fd-49ac-830d-989a5482057c', 'type': 'project', 'uuid': 'a5dc0037-9655-44b1-8af6-f013fea66ee0'} 09/01/2025 01:35:11.505 7fbb3cfbbd60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.506 7fbb3cfbbd60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system elements = { application_policy_set_refs : [{"to": ["default-domain", "k8s-kube-system", "default-application-policy-set"], "attr": null, "uuid": "ec38d33f-4d7d-417f-97ef-9c2c99484343"}]display_name : "k8s-kube-system"fq_name : ["default-domain", "k8s-kube-system"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 11951427749879825585, "uuid_lslong": 10013454790172241632}, "enable": true, "created": "2025-09-01T01:34:55.985402", "last_modified": "2025-09-01T01:35:11.495489", "description": null, "user_visible": true, "creator": null}parent_type : "domain"parent_uuid : "ce056a4b-12ea-41a5-896d-393d506f2011"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0" } deleted = False >> 09/01/2025 01:35:11.506 7fbb3cfbbd60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_objects in resource project: 'security_logging_objects' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_groups in resource project: 'security_groups' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_networks in resource project: 'virtual_networks' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_configs in resource project: 'qos_configs' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipams in resource project: 'network_ipams' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policys in resource project: 'network_policys' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource project: 'virtual_machine_interfaces' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_services in resource project: 'bgp_as_a_services' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policys in resource project: 'routing_policys' 09/01/2025 01:35:11.507 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregates in resource project: 'route_aggregates' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instances in resource project: 'service_instances' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_checks in resource project: 'service_health_checks' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_tables in resource project: 'route_tables' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_tables in resource project: 'interface_route_tables' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_routers in resource project: 'logical_routers' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource project: 'api_access_lists' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policys in resource project: 'multicast_policys' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pools in resource project: 'loadbalancer_pools' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitors in resource project: 'loadbalancer_healthmonitors' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ips in resource project: 'virtual_ips' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listeners in resource project: 'loadbalancer_listeners' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancers in resource project: 'loadbalancers' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpns in resource project: 'bgpvpns' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource project: 'alarms' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_managements in resource project: 'policy_managements' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource project: 'service_groups' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource project: 'address_groups' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource project: 'firewall_rules' 09/01/2025 01:35:11.508 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource project: 'firewall_policys' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource project: 'application_policy_sets' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tags in resource project: 'tags' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_groups in resource project: 'device_functional_groups' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_groups in resource project: 'virtual_port_groups' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profiles in resource project: 'telemetry_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profiles in resource project: 'sflow_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profiles in resource project: 'grpc_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profiles in resource project: 'snmp_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profiles in resource project: 'netconf_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profiles in resource project: 'storm_control_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profiles in resource project: 'port_profiles' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_services in resource project: 'host_based_services' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_configs in resource project: 'structured_syslog_configs' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_refs in resource project: 'namespace_refs' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_refs in resource project: 'floating_ip_pool_refs' 09/01/2025 01:35:11.509 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_refs in resource project: 'alias_ip_pool_refs' 09/01/2025 01:35:11.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource project: 'tag_refs' 09/01/2025 01:35:11.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource project: 'floating_ip_back_refs' 09/01/2025 01:35:11.510 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource project: 'alias_ip_back_refs' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:11.519 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.520 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:11.521 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:11.575 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:11.576 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:11.582 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d650e01f-39d4-4eb1-91f8-ca54373fcba6 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-pod-network url = http://10.0.0.50:8082/virtual-network/d650e01f-39d4-4eb1-91f8-ca54373fcba6 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}, {"tenant": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:11.592 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-1e81e314-b72a-41a5-9d7a-1d919389fb88', 'type': 'virtual_network', 'uuid': 'd650e01f-39d4-4eb1-91f8-ca54373fcba6'} 09/01/2025 01:35:11.598 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.599 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-pod-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-pod-network"fabric_snat : truefq_name : ["default-domain", "k8s-default", "k8s-default-pod-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15443089546972253873, "uuid_lslong": 10518379392776784806}, "enable": true, "created": "2025-09-01T01:34:56.642429", "last_modified": "2025-09-01T01:35:11.587296", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-pod-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "eada5f2d-448e-4621-9fea-812925eb46e9"}], "host_routes": null}, "uuid": "4f73584c-440b-479b-91ff-c256c5407f1c"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}, {"tenant": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "d650e01f-39d4-4eb1-91f8-ca54373fcba6"virtual_network_network_id : 7virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:11.599 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.608 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 5fa5f383-4684-4ec5-88cc-18491581bc2e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-pod-network:k8s-default-pod-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:11.619 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-pod-network', 'k8s-default-pod-network'], 'oper': 'UPDATE', 'request-id': 'req-1e81e314-b72a-41a5-9d7a-1d919389fb88', 'type': 'routing_instance', 'uuid': '5fa5f383-4684-4ec5-88cc-18491581bc2e'} 09/01/2025 01:35:11.660 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a6c41605-29b5-4568-a40b-17e3f103c269 object_type = virtual_network identifier_name = default-domain:k8s-default:k8s-default-service-network url = http://10.0.0.50:8082/virtual-network/a6c41605-29b5-4568-a40b-17e3f103c269 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"virtual-network": {"uuid": "a6c41605-29b5-4568-a40b-17e3f103c269", "perms2": {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}, {"tenant": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "tenant_access": 4}]}}} domain = default-domain >> 09/01/2025 01:35:11.672 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_vxlan_network_identifier': None}, 'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-e53f458c-8756-4e94-bbf6-1fa599f7e4b4', 'type': 'virtual_network', 'uuid': 'a6c41605-29b5-4568-a40b-17e3f103c269'} 09/01/2025 01:35:11.680 7fbb3df45900 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.680 7fbb3df45900 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-default:k8s-default-service-network elements = { address_allocation_mode : "flat-subnet-only"display_name : "k8s-default-service-network"fabric_snat : falsefq_name : ["default-domain", "k8s-default", "k8s-default-service-network"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12016753917161719144, "uuid_lslong": 11820567914732569193}, "enable": true, "created": "2025-09-01T01:34:57.045929", "last_modified": "2025-09-01T01:35:11.665939", "description": null, "user_visible": true, "creator": null}is_shared : falsenetwork_ipam_refs : [{"to": ["default-domain", "k8s-default", "k8s-service-ipam"], "attr": {"ipam_subnets": [{"subnet_uuid": "cdf5a7bb-3fee-45f9-9299-6a9e2ef0e400"}], "host_routes": null}, "uuid": "f614050a-708e-4299-938e-45ace122514b"}]network_policy_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "1f1a62da-5984-46dc-82ee-e07966ea5188"}, {"to": ["default-domain", "k8s-default", "k8s-default-ip-fabric-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "5f5d49e0-6781-418e-8849-6d0cb836c60b"}, {"to": ["default-domain", "k8s-default", "k8s-default-service-np"], "attr": {"sequence": {"major": 0, "minor": 0}, "timer": null}, "uuid": "a6209b87-6082-4775-b38a-86ce0eab279a"}]parent_type : "project"parent_uuid : "9023d5fc-34ad-4e7e-9d9e-872a06007bf8"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": [{"tenant": "081306e5-ff47-439a-8b16-a4f2e0b3335a", "tenant_access": 4}, {"tenant": "8e0d7a3c-989c-4add-b7cb-c1b895a01899", "tenant_access": 4}, {"tenant": "0eb5a575-e09c-4708-abd9-312cfc4e85f8", "tenant_access": 4}, {"tenant": "a5dc0037-9655-44b1-8af6-f013fea66ee0", "tenant_access": 4}]}tag_refs : [{"to": ["application=k8s"], "attr": null, "uuid": "2987d33f-67f3-4bad-a5bc-d9b567026005"}]uuid : "a6c41605-29b5-4568-a40b-17e3f103c269"virtual_network_network_id : 8virtual_network_properties : {"allow_transit": null, "network_id": null, "vxlan_network_identifier": null, "forwarding_mode": "l3", "rpf": null, "mirror_destination": false, "max_flows": 0, "mtu": null} } deleted = False >> 09/01/2025 01:35:11.680 7fbb3df45900 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:11.683 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a099888b-9d11-4587-8b98-4d1673309c9e object_type = routing_instance identifier_name = default-domain:k8s-default:k8s-default-service-network:k8s-default-service-network url = http://127.0.0.1/routing-instances operation = http_put domain = default-domain >> 09/01/2025 01:35:11.696 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-default', 'k8s-default-service-network', 'k8s-default-service-network'], 'oper': 'UPDATE', 'request-id': 'req-e53f458c-8756-4e94-bbf6-1fa599f7e4b4', 'type': 'routing_instance', 'uuid': 'a099888b-9d11-4587-8b98-4d1673309c9e'} 09/01/2025 01:35:11.752 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e0fa3e04-8145-47f6-9b69-718272132295 object_type = security_group identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg url = http://10.0.0.50:8082/security-group/e0fa3e04-8145-47f6-9b69-718272132295 operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"security-group": {"uuid": "e0fa3e04-8145-47f6-9b69-718272132295", "fq_name": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "parent_type": "project", "security_group_entries": {"policy_rule": [{"rule_sequence": null, "rule_uuid": "666cafef-a52a-4e0f-ae65-5a3223d17eec", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "27fd452f-507e-42c2-a252-633011958af2", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "5577cf12-24bd-4fdf-aed4-6880b57a5164", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv4", "created": null, "last_modified": null, "description": null}, {"rule_sequence": null, "rule_uuid": "8e1e609d-e886-4251-b155-66d63910c643", "direction": ">", "protocol": "any", "src_addresses": [{"subnet": null, "virtual_network": null, "security_group": "local", "network_policy": null, "subnet_list": []}], "src_ports": [{"start_port": 0, "end_port": 65535}], "application": [], "dst_addresses": [{"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}], "dst_ports": [{"start_port": 0, "end_port": 65535}], "action_list": null, "ethertype": "IPv6", "created": null, "last_modified": null, "description": null}]}, "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16211337997743114230, "uuid_lslong": 11198606753506992789}, "enable": true, "created": "2025-09-01T01:34:56.343303", "last_modified": "2025-09-01T01:34:56.428213", "description": "Default security group", "user_visible": true, "creator": null}}} domain = default-domain >> 09/01/2025 01:35:11.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:11.827 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-c5c73fdb-c44a-450f-a44a-0ad4bdee763f', 'type': 'security_group', 'uuid': 'e0fa3e04-8145-47f6-9b69-718272132295'} 09/01/2025 01:35:11.832 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e0fa3e04-8145-47f6-9b69-718272132295 object_type = security_group identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg url = http://10.0.0.50:8082/prop-collection-update operation = prop-collection-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"uuid": "e0fa3e04-8145-47f6-9b69-718272132295", "updates": [{"field": "annotations", "operation": "set", "value": {"key": "kind", "value": "Namespace"}, "position": "kind"}, {"field": "annotations", "operation": "set", "value": {"key": "owner", "value": "k8s"}, "position": "owner"}, {"field": "annotations", "operation": "set", "value": {"key": "cluster", "value": "k8s"}, "position": "cluster"}, {"field": "annotations", "operation": "set", "value": {"key": "project", "value": "k8s-kube-system"}, "position": "project"}, {"field": "annotations", "operation": "set", "value": {"key": "namespace", "value": "kube-system"}, "position": "namespace"}, {"field": "annotations", "operation": "set", "value": {"key": "name", "value": "k8s-kube-system-default-sg"}, "position": "name"}]} domain = default-domain >> 09/01/2025 01:35:11.841 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 299bf655-b7a5-41e2-877c-35bccfef2eda object_type = access_control_list identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg:ingress-access-control-list url = http://10.0.0.50:8082/access-control-list/299bf655-b7a5-41e2-877c-35bccfef2eda operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "299bf655-b7a5-41e2-877c-35bccfef2eda", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "666cafef-a52a-4e0f-ae65-5a3223d17eec", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "27fd452f-507e-42c2-a252-633011958af2", "direction": null}]}, "access_control_list_hash": 6313896201487607181}} domain = default-domain >> 09/01/2025 01:35:11.854 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'extra_dict': {'deallocated_security_group_id': None}, 'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg'], 'oper': 'UPDATE', 'request-id': 'req-f4377f5a-1796-4fa8-b1ea-fb62420a7ca8', 'type': 'security_group', 'uuid': 'e0fa3e04-8145-47f6-9b69-718272132295'} 09/01/2025 01:35:11.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:11.860 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:11.869 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'ingress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-f66c4fd0-b770-4d76-99e7-c8fd6bbfe945', 'type': 'access_control_list', 'uuid': '299bf655-b7a5-41e2-877c-35bccfef2eda'} 09/01/2025 01:35:11.872 7fbb3d1f6180 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 448ddfc1-0480-4786-9b61-9816732b043b object_type = access_control_list identifier_name = default-domain:k8s-kube-system:k8s-kube-system-default-sg:egress-access-control-list url = http://10.0.0.50:8082/access-control-list/448ddfc1-0480-4786-9b61-9816732b043b operation = http_put useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-schema remote_ip = 10.0.0.50 body = {"access-control-list": {"uuid": "448ddfc1-0480-4786-9b61-9816732b043b", "access_control_list_entries": {"dynamic": null, "acl_rule": [{"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "0.0.0.0", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv4"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "5577cf12-24bd-4fdf-aed4-6880b57a5164", "direction": null}, {"match_condition": {"protocol": "any", "src_address": {"subnet": null, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "src_port": {"start_port": 0, "end_port": 65535}, "dst_address": {"subnet": {"ip_prefix": "::", "ip_prefix_len": 0}, "virtual_network": null, "security_group": null, "network_policy": null, "subnet_list": []}, "dst_port": {"start_port": 0, "end_port": 65535}, "ethertype": "IPv6"}, "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "rule_uuid": "8e1e609d-e886-4251-b155-66d63910c643", "direction": null}]}, "access_control_list_hash": -7333639033842153749}} domain = default-domain >> 09/01/2025 01:35:11.881 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'k8s-kube-system-default-sg', 'egress-access-control-list'], 'oper': 'UPDATE', 'request-id': 'req-1b59da51-eaa3-4e2f-8896-b8802e3a3132', 'type': 'access_control_list', 'uuid': '448ddfc1-0480-4786-9b61-9816732b043b'} 09/01/2025 01:35:11.910 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:11.910 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:11.910 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:11.910 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:11.911 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:11.911 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:11.911 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:11.967 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:11.968 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:11.969 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:11.970 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:11.971 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:11.972 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:11.973 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:11.974 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:11.975 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:11.976 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:11.977 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:11.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:11.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:11.978 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:11.982 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-system-ingress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-system-ingress 09/01/2025 01:35:11.985 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-system-ingress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-system-ingress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-system"], "tag_ids": [], "any": null}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-system-ingress"}} domain = default-domain >> 09/01/2025 01:35:12.063 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-system-ingress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-system-ingress', 'endpoint_1': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842752], 'tags': ['namespace=kube-system'], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-system-ingress'], 'id_perms': {'created': '2025-09-01T01:35:12.005815', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:12.005815', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12917800681854254437, 'uuid_mslong': 9121708868258582194}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-system'], 'uuid': '317d6be2-b0aa-4cee-87d2-56074b8ba3b1'}], 'uuid': '7e96d1e3-0515-4eb2-b345-3f475938bd65'}, 'oper': 'CREATE', 'request-id': 'req-a571d2ec-12f7-40d9-a141-8aa030568bcc', 'type': 'firewall_rule', 'uuid': '7e96d1e3-0515-4eb2-b345-3f475938bd65'} 09/01/2025 01:35:12.064 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:12.064 7fbb3df45400 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.065 7fbb3df45400 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-system-ingress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-system-ingress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-system"], "tag_ids": [16842752], "any": null}fq_name : ["default-policy-management", "k8s-Namespace-kube-system-ingress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 9121708868258582194, "uuid_lslong": 12917800681854254437}, "enable": true, "created": "2025-09-01T01:35:12.005815", "last_modified": "2025-09-01T01:35:12.005815", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-system"], "uuid": "317d6be2-b0aa-4cee-87d2-56074b8ba3b1"}]uuid : "7e96d1e3-0515-4eb2-b345-3f475938bd65" } deleted = False >> 09/01/2025 01:35:12.065 7fbb3df45400 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.099 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:12.099 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:12.100 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:12.100 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:12.100 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:12.106 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:12.106 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:12.106 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:12.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:12.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:12.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:12.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:12.110 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:12.114 7fbb3df45a40 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:35:12.116 7fbb3df45a40 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:35:12.162 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "7e96d1e3-0515-4eb2-b345-3f475938bd65", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-system-ingress"], "operation": "ADD", "attr": {"sequence": "00009.0"}} domain = default-domain >> 09/01/2025 01:35:12.197 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-cd5f9f37-b7fe-4e40-bd9d-4d0cf7581682', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:12.201 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.202 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-egress"], "attr": {"sequence": "00008.0"}, "uuid": "659c4eac-56d7-4412-b013-feda7ae8db07"}, {"to": ["default-policy-management", "k8s-Namespace-kube-system-ingress"], "attr": {"sequence": "00009.0"}, "uuid": "7e96d1e3-0515-4eb2-b345-3f475938bd65"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "attr": {"sequence": "00007.0"}, "uuid": "b70784bf-913f-4579-9336-91b517b32240"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "attr": {"sequence": "00006.0"}, "uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:12.192716", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:12.202 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.213 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_refs in resource application-policy-set: 'global_vrouter_config_refs' 09/01/2025 01:35:12.213 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource application-policy-set: 'project_back_refs' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource policy-management: 'parent_type' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_groups in resource policy-management: 'service_groups' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_groups in resource policy-management: 'address_groups' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rules in resource policy-management: 'firewall_rules' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policys in resource policy-management: 'firewall_policys' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_sets in resource policy-management: 'application_policy_sets' 09/01/2025 01:35:12.267 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource policy-management: 'tag_refs' 09/01/2025 01:35:12.322 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:12.323 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:12.324 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:12.325 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:12.326 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:12.327 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:12.328 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:12.329 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:12.330 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:12.335 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-policy-management', 'k8s-Namespace-kube-system-egress'] not found in DB: Unknown id: firewall_rule default-policy-management:k8s-Namespace-kube-system-egress 09/01/2025 01:35:12.392 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = firewall_rule identifier_name = default-policy-management:k8s-Namespace-kube-system-egress url = http://10.0.0.50:8082/firewall-rules operation = post useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"firewall-rule": {"uuid": null, "fq_name": ["default-policy-management", "k8s-Namespace-kube-system-egress"], "parent_type": "policy-management", "action_list": {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}, "service": {"protocol": "any", "protocol_id": null, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}, "endpoint_1": {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-system"], "tag_ids": [], "any": null}, "endpoint_2": {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}, "direction": ">", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "k8s-Namespace-kube-system-egress"}} domain = default-domain >> 09/01/2025 01:35:12.525 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-Namespace-kube-system-egress'], 'obj_dict': {'action_list': {'alert': False, 'apply_service': [], 'assign_routing_instance': None, 'gateway_name': None, 'host_based_service': False, 'log': False, 'mirror_to': None, 'qos_action': None, 'service_properties': None, 'simple_action': 'pass'}, 'direction': '>', 'display_name': 'k8s-Namespace-kube-system-egress', 'endpoint_1': {'address_group': None, 'any': None, 'subnet': None, 'tag_ids': [16842752], 'tags': ['namespace=kube-system'], 'virtual_network': None}, 'endpoint_2': {'address_group': None, 'any': True, 'subnet': None, 'tag_ids': [], 'tags': [], 'virtual_network': None}, 'fq_name': ['default-policy-management', 'k8s-Namespace-kube-system-egress'], 'id_perms': {'created': '2025-09-01T01:35:12.470619', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:12.470619', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 11622934083351091608, 'uuid_mslong': 14421497729005404891}}, 'match_tag_types': {'tag_type': [1]}, 'parent_type': 'policy-management', 'parent_uuid': '4a4fd292-14a2-4148-9861-3579ce70f727', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'service': {'dst_ports': {'end_port': 65535, 'start_port': 0}, 'protocol': 'any', 'protocol_id': 0, 'src_ports': None}, 'tag_refs': [{'to': ['namespace=kube-system'], 'uuid': '317d6be2-b0aa-4cee-87d2-56074b8ba3b1'}], 'uuid': 'c82373c6-b278-42db-a14c-f4f7c5ec1998'}, 'oper': 'CREATE', 'request-id': 'req-fa8d65a1-5ce9-40f7-90e7-60e71d67ac20', 'type': 'firewall_rule', 'uuid': 'c82373c6-b278-42db-a14c-f4f7c5ec1998'} 09/01/2025 01:35:12.525 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectFirewallRuleTable:ContrailConfigTrace] map 09/01/2025 01:35:12.526 7fbb3df45a40 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.526 7fbb3df45a40 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-Namespace-kube-system-egress elements = { action_list : {"simple_action": "pass", "gateway_name": null, "apply_service": [], "service_properties": null, "mirror_to": null, "assign_routing_instance": null, "log": false, "alert": false, "qos_action": null, "host_based_service": false}direction : ">"display_name : "k8s-Namespace-kube-system-egress"endpoint_1 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": ["namespace=kube-system"], "tag_ids": [16842752], "any": null}endpoint_2 : {"subnet": null, "virtual_network": null, "address_group": null, "tags": [], "tag_ids": [], "any": true}fq_name : ["default-policy-management", "k8s-Namespace-kube-system-egress"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 14421497729005404891, "uuid_lslong": 11622934083351091608}, "enable": true, "created": "2025-09-01T01:35:12.470619", "last_modified": "2025-09-01T01:35:12.470619", "description": null, "user_visible": true, "creator": null}match_tag_types : {"tag_type": [1]}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}service : {"protocol": "any", "protocol_id": 0, "src_ports": null, "dst_ports": {"start_port": 0, "end_port": 65535}}tag_refs : [{"to": ["namespace=kube-system"], "uuid": "317d6be2-b0aa-4cee-87d2-56074b8ba3b1"}]uuid : "c82373c6-b278-42db-a14c-f4f7c5ec1998" } deleted = False >> 09/01/2025 01:35:12.526 7fbb3df45a40 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.553 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:12.553 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:12.553 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:12.553 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:12.553 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:12.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-policy: 'security_logging_object_refs' 09/01/2025 01:35:12.602 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource firewall-policy: 'tag_refs' 09/01/2025 01:35:12.603 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource firewall-policy: 'application_policy_set_back_refs' 09/01/2025 01:35:12.608 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_refs in resource firewall-rule: 'service_group_refs' 09/01/2025 01:35:12.608 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_refs in resource firewall-rule: 'address_group_refs' 09/01/2025 01:35:12.608 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource firewall-rule: 'virtual_network_refs' 09/01/2025 01:35:12.608 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource firewall-rule: 'security_logging_object_refs' 09/01/2025 01:35:12.608 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource firewall-rule: 'firewall_policy_back_refs' 09/01/2025 01:35:12.616 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = aebd7f43-362b-4543-a3a3-c56771bd18d7 object_type = firewall_policy identifier_name = default-policy-management:k8s-allowall url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "firewall-policy", "uuid": "aebd7f43-362b-4543-a3a3-c56771bd18d7", "ref-type": "firewall-rule", "ref-uuid": "c82373c6-b278-42db-a14c-f4f7c5ec1998", "ref-fq-name": ["default-policy-management", "k8s-Namespace-kube-system-egress"], "operation": "ADD", "attr": {"sequence": "00010.0"}} domain = default-domain >> 09/01/2025 01:35:12.662 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-policy-management', 'k8s-allowall'], 'oper': 'UPDATE', 'request-id': 'req-cc473e95-bceb-4aee-a9bb-72ffb67a594f', 'type': 'firewall_policy', 'uuid': 'aebd7f43-362b-4543-a3a3-c56771bd18d7'} 09/01/2025 01:35:12.667 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.668 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-policy-management:k8s-allowall elements = { annotations : {"key_value_pair": [{"key": "after_tail", "value": "True"}, {"key": "cluster", "value": "k8s"}, {"key": "k8s_uuid", "value": null}, {"key": "kind", "value": "VncSecurityPolicy"}, {"key": "name", "value": "allowall"}, {"key": "namespace", "value": null}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": null}, {"key": "spec", "value": "null"}]}display_name : "k8s-allowall"firewall_rule_refs : [{"to": ["default-policy-management", "k8s-Namespace-contrail-ingress"], "attr": {"sequence": "00001.0"}, "uuid": "4d5f5095-e368-4151-8d53-92634c6a84f3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-egress"], "attr": {"sequence": "00008.0"}, "uuid": "659c4eac-56d7-4412-b013-feda7ae8db07"}, {"to": ["default-policy-management", "k8s-Namespace-kube-system-ingress"], "attr": {"sequence": "00009.0"}, "uuid": "7e96d1e3-0515-4eb2-b345-3f475938bd65"}, {"to": ["default-policy-management", "k8s-Namespace-default-ingress"], "attr": {"sequence": "00003.0"}, "uuid": "93d20365-6161-478d-b8e7-131cf2e827ab"}, {"to": ["default-policy-management", "k8s-Namespace-contrail-egress"], "attr": {"sequence": "00002.0"}, "uuid": "b1578148-cb6e-4ef6-a10a-3911dfe13f1b"}, {"to": ["default-policy-management", "k8s-Namespace-kube-public-ingress"], "attr": {"sequence": "00007.0"}, "uuid": "b70784bf-913f-4579-9336-91b517b32240"}, {"to": ["default-policy-management", "k8s-Namespace-kube-system-egress"], "attr": {"sequence": "00010.0"}, "uuid": "c82373c6-b278-42db-a14c-f4f7c5ec1998"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-egress"], "attr": {"sequence": "00006.0"}, "uuid": "df980d9f-8014-47cc-a200-84ae887cc6c3"}, {"to": ["default-policy-management", "k8s-Namespace-kube-node-lease-ingress"], "attr": {"sequence": "00005.0"}, "uuid": "f1827da2-8bae-4c45-87e0-fa503064e9e9"}, {"to": ["default-policy-management", "k8s-Namespace-default-egress"], "attr": {"sequence": "00004.0"}, "uuid": "f698f012-e2b4-4355-a731-5370f71b8f8b"}]fq_name : ["default-policy-management", "k8s-allowall"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12591360059846116675, "uuid_lslong": 11791485297466611927}, "enable": true, "created": "2025-09-01T01:34:58.612519", "last_modified": "2025-09-01T01:35:12.657829", "description": null, "user_visible": true, "creator": null}parent_type : "policy-management"parent_uuid : "4a4fd292-14a2-4148-9861-3579ce70f727"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "aebd7f43-362b-4543-a3a3-c56771bd18d7" } deleted = False >> 09/01/2025 01:35:12.668 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.722 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource tag: 'parent_type' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource tag: 'tag_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flag_back_refs in resource tag: 'feature_flag_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_system_config_back_refs in resource tag: 'global_system_config_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_root_back_refs in resource tag: 'config_root_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource tag: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_back_refs in resource tag: 'bgp_router_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zone_back_refs in resource tag: 'control_node_zone_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_back_refs in resource tag: 'security_logging_object_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_config_back_refs in resource tag: 'global_vrouter_config_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource tag: 'virtual_network_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource tag: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field project_back_refs in resource tag: 'project_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_policy_back_refs in resource tag: 'network_policy_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_back_refs in resource tag: 'security_group_back_refs' 09/01/2025 01:35:12.723 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_config_back_refs in resource tag: 'global_qos_config_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource tag: 'qos_config_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field domain_back_refs in resource tag: 'domain_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field namespace_back_refs in resource tag: 'namespace_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field forwarding_class_back_refs in resource tag: 'forwarding_class_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_queue_back_refs in resource tag: 'qos_queue_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_back_refs in resource tag: 'bgpvpn_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource tag: 'logical_router_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_back_refs in resource tag: 'network_ipam_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_router_back_refs in resource tag: 'virtual_router_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_list_back_refs in resource tag: 'access_control_list_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_back_refs in resource tag: 'virtual_machine_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource tag: 'routing_instance_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource tag: 'instance_ip_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource tag: 'physical_router_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource tag: 'logical_interface_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource tag: 'flow_node_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource tag: 'subnet_back_refs' 09/01/2025 01:35:12.724 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pool_back_refs in resource tag: 'floating_ip_pool_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource tag: 'floating_ip_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pool_back_refs in resource tag: 'alias_ip_pool_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource tag: 'alias_ip_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_back_refs in resource tag: 'sub_cluster_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource tag: 'routing_policy_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource tag: 'route_aggregate_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource tag: 'customer_attachment_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource tag: 'provider_attachment_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_instance_back_refs in resource tag: 'service_instance_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_template_back_refs in resource tag: 'service_template_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_back_refs in resource tag: 'service_health_check_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_back_refs in resource tag: 'interface_route_table_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource tag: 'port_tuple_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_back_refs in resource tag: 'virtual_DNS_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_DNS_record_back_refs in resource tag: 'virtual_DNS_record_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_back_refs in resource tag: 'route_table_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_back_refs in resource tag: 'route_target_back_refs' 09/01/2025 01:35:12.725 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource tag: 'fabric_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_node_back_refs in resource tag: 'config_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_node_back_refs in resource tag: 'analytics_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_node_back_refs in resource tag: 'devicemgr_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_node_back_refs in resource tag: 'database_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_node_back_refs in resource tag: 'webui_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_node_back_refs in resource tag: 'config_database_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_node_back_refs in resource tag: 'analytics_alarm_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_node_back_refs in resource tag: 'analytics_snmp_node_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_set_back_refs in resource tag: 'service_appliance_set_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_back_refs in resource tag: 'service_appliance_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_back_refs in resource tag: 'physical_interface_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_list_back_refs in resource tag: 'api_access_list_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field dsa_rule_back_refs in resource tag: 'dsa_rule_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field discovery_service_assignment_back_refs in resource tag: 'discovery_service_assignment_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_back_refs in resource tag: 'bridge_domain_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_back_refs in resource tag: 'multicast_policy_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_back_refs in resource tag: 'service_endpoint_back_refs' 09/01/2025 01:35:12.726 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_connection_module_back_refs in resource tag: 'service_connection_module_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_object_back_refs in resource tag: 'service_object_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_device_config_back_refs in resource tag: 'network_device_config_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field e2_service_provider_back_refs in resource tag: 'e2_service_provider_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field peering_policy_back_refs in resource tag: 'peering_policy_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource tag: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_listener_back_refs in resource tag: 'loadbalancer_listener_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_member_back_refs in resource tag: 'loadbalancer_member_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_healthmonitor_back_refs in resource tag: 'loadbalancer_healthmonitor_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource tag: 'virtual_ip_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource tag: 'loadbalancer_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarm_back_refs in resource tag: 'alarm_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_group_back_refs in resource tag: 'service_group_back_refs' 09/01/2025 01:35:12.727 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field address_group_back_refs in resource tag: 'address_group_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource tag: 'firewall_rule_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_policy_back_refs in resource tag: 'firewall_policy_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field application_policy_set_back_refs in resource tag: 'application_policy_set_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field policy_management_back_refs in resource tag: 'policy_management_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_type_back_refs in resource tag: 'tag_type_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_back_refs in resource tag: 'tag_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_properties_back_refs in resource tag: 'config_properties_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_template_back_refs in resource tag: 'job_template_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource tag: 'data_center_interconnect_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_back_refs in resource tag: 'intent_map_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_namespace_back_refs in resource tag: 'fabric_namespace_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_back_refs in resource tag: 'hardware_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field card_back_refs in resource tag: 'card_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_chassis_back_refs in resource tag: 'device_chassis_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profile_back_refs in resource tag: 'node_profile_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_functional_group_back_refs in resource tag: 'device_functional_group_back_refs' 09/01/2025 01:35:12.728 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definition_back_refs in resource tag: 'role_definition_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_config_back_refs in resource tag: 'role_config_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_role_back_refs in resource tag: 'physical_role_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_image_back_refs in resource tag: 'device_image_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource tag: 'link_aggregation_group_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_role_back_refs in resource tag: 'overlay_role_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field hardware_inventory_back_refs in resource tag: 'hardware_inventory_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field cli_config_back_refs in resource tag: 'cli_config_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource tag: 'virtual_port_group_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_back_refs in resource tag: 'node_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_back_refs in resource tag: 'port_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_group_back_refs in resource tag: 'port_group_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_back_refs in resource tag: 'feature_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_config_back_refs in resource tag: 'feature_config_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field telemetry_profile_back_refs in resource tag: 'telemetry_profile_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sflow_profile_back_refs in resource tag: 'sflow_profile_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field grpc_profile_back_refs in resource tag: 'grpc_profile_back_refs' 09/01/2025 01:35:12.729 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field netconf_profile_back_refs in resource tag: 'netconf_profile_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field snmp_profile_back_refs in resource tag: 'snmp_profile_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field storm_control_profile_back_refs in resource tag: 'storm_control_profile_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_back_refs in resource tag: 'port_profile_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource tag: 'host_based_service_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_config_back_refs in resource tag: 'global_analytics_config_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_config_back_refs in resource tag: 'structured_syslog_config_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_message_back_refs in resource tag: 'structured_syslog_message_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_hostname_record_back_refs in resource tag: 'structured_syslog_hostname_record_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_tenant_record_back_refs in resource tag: 'structured_syslog_tenant_record_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_application_record_back_refs in resource tag: 'structured_syslog_application_record_back_refs' 09/01/2025 01:35:12.730 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field structured_syslog_sla_profile_back_refs in resource tag: 'structured_syslog_sla_profile_back_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:12.780 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:35:12.781 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:35:12.788 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = e02048c4-86d3-11f0-93e1-fa163e599e0d object_type = virtual_machine_interface identifier_name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d url = http://127.0.0.1/virtual-machine-interfaces operation = http_put domain = default-domain >> 09/01/2025 01:35:12.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:12.812 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-domain', 'k8s-kube-system', 'coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d'], 'oper': 'UPDATE', 'request-id': 'req-fd135d25-ba94-42ee-a11e-0478c91d7bdf', 'type': 'virtual_machine_interface', 'uuid': 'e02048c4-86d3-11f0-93e1-fa163e599e0d'} 09/01/2025 01:35:12.814 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:12.815 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-domain:k8s-kube-system:coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d elements = { annotations : {"key_value_pair": [{"key": "cluster", "value": "k8s"}, {"key": "index", "value": "0/1"}, {"key": "kind", "value": "Pod"}, {"key": "name", "value": "coredns-6f867cd986-wlkqx"}, {"key": "namespace", "value": "kube-system"}, {"key": "network", "value": "default"}, {"key": "owner", "value": "k8s"}, {"key": "project", "value": "k8s-kube-system"}]}display_name : "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"fq_name : ["default-domain", "k8s-kube-system", "coredns-6f867cd986-wlkqx__e02048c4-86d3-11f0-93e1-fa163e599e0d"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 16149988272663368176, "uuid_lslong": 10656073166777589261}, "enable": true, "created": "2025-09-01T01:35:00.522540", "last_modified": "2025-09-01T01:35:12.792507", "description": null, "user_visible": true, "creator": null}parent_type : "project"parent_uuid : "a5dc0037-9655-44b1-8af6-f013fea66ee0"perms2 : {"owner": "None", "owner_access": 7, "global_access": 0, "share": []}port_security_enabled : truerouting_instance_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network", "k8s-default-pod-network"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "5fa5f383-4684-4ec5-88cc-18491581bc2e"}]security_group_refs : [{"to": ["default-domain", "k8s-kube-system", "k8s-kube-system-default-sg"], "attr": null, "uuid": "e0fa3e04-8145-47f6-9b69-718272132295"}]tag_refs : [{"to": ["namespace=kube-system"], "attr": null, "uuid": "317d6be2-b0aa-4cee-87d2-56074b8ba3b1"}, {"to": ["pod-template-hash=6f867cd986"], "attr": null, "uuid": "5877f896-8829-4ac4-bf58-e4f1ee415b51"}, {"to": ["k8s-app=kube-dns"], "attr": null, "uuid": "6cc1ea02-8199-48e3-9f31-a8a523d1d09f"}, {"to": ["kubernetes.io__slash__metadata.name=kube-system"], "attr": null, "uuid": "76a38ee9-bbe3-4406-bfac-c5c345aefbe5"}]uuid : "e02048c4-86d3-11f0-93e1-fa163e599e0d"virtual_machine_interface_mac_addresses : {"mac_address": ["02:e0:20:48:c4:86"]}virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "attr": null, "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_network_refs : [{"to": ["default-domain", "k8s-default", "k8s-default-pod-network"], "attr": null, "uuid": "d650e01f-39d4-4eb1-91f8-ca54373fcba6"}] } deleted = False >> 09/01/2025 01:35:12.815 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:13.189 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:13.190 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:13.190 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:13.190 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:13.190 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:13.191 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:13.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:14.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:15.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:16.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:17.191 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:17.192 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:17.192 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:17.192 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:17.193 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:18.740 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:19.742 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:20.743 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:21.193 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:21.194 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:21.194 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:21.194 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:21.194 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:21.195 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:21.744 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:22.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:23.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:24.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:25.195 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:25.196 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:25.197 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:25.197 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:25.197 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:25.197 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:25.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:26.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:27.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:28.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:29.197 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:29.198 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:29.198 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:29.198 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:29.199 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:29.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:30.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:31.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:32.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:33.200 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:33.201 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:33.201 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:33.202 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:33.203 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:33.203 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:33.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:34.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:35.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:36.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:37.202 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:37.204 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:37.204 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:37.204 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:37.204 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:37.204 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:37.205 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:37.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:38.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:39.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:40.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:41.205 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:41.206 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:41.206 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:41.206 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:41.207 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:41.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:42.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:43.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:44.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:45.207 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:45.208 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:45.209 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:45.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:46.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:48.740 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:49.209 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:49.210 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:49.211 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:35:49.439 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:35:49.440 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:35:49.441 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:35:49.441 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:35:49.489 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 09/01/2025 01:35:49.490 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 09/01/2025 01:35:49.494 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = 70eaa5ad-608f-40c9-a193-2be391842efd object_type = global_system_config identifier_name = default-global-system-config url = http://10.0.0.50:8082/global-system-config/70eaa5ad-608f-40c9-a193-2be391842efd operation = http_put useragent = an-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.22 body = {"global-system-config": {"uuid": "70eaa5ad-608f-40c9-a193-2be391842efd", "ip_fabric_subnets": {"subnet": [{"ip_prefix": "10.0.0.0", "ip_prefix_len": 24}]}}} domain = default-domain >> 09/01/2025 01:35:49.502 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config'], 'oper': 'UPDATE', 'request-id': 'req-5c9a772d-a744-45b7-80ef-09a53d7ad23d', 'type': 'global_system_config', 'uuid': '70eaa5ad-608f-40c9-a193-2be391842efd'} 09/01/2025 01:35:49.554 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1'] not found in DB: Unknown id: virtual_router default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1 09/01/2025 01:35:49.558 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_router identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/virtual-routers operation = post useragent = an-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.22 body = {"virtual-router": {"uuid": null, "fq_name": ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1"], "parent_type": "global-system-config", "virtual_router_dpdk_enabled": false, "virtual_router_ip_address": "10.0.0.22", "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "an-jenkins-deploy-platform-juju-k8s-1454-1"}} domain = default-domain >> 09/01/2025 01:35:49.575 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1'], 'obj_dict': {'display_name': 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1'], 'id_perms': {'created': '2025-09-01T01:35:49.567966', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:49.567966', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 10794658036228347919, 'uuid_mslong': 12163681958422269707}}, 'parent_type': 'global-system-config', 'parent_uuid': '70eaa5ad-608f-40c9-a193-2be391842efd', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'a8ce144b-9496-4f0b-95ce-544a9934d80f', 'virtual_router_dpdk_enabled': False, 'virtual_router_ip_address': '10.0.0.22'}, 'oper': 'CREATE', 'request-id': 'req-ee08329c-5872-4f7a-b645-2f27e34a25a4', 'type': 'virtual_router', 'uuid': 'a8ce144b-9496-4f0b-95ce-544a9934d80f'} 09/01/2025 01:35:49.575 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVRouter:ContrailConfigTrace] map 09/01/2025 01:35:49.576 7fbb3e0270e0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.576 7fbb3e0270e0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = an-jenkins-deploy-platform-juju-k8s-1454-1 elements = { display_name : "an-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12163681958422269707, "uuid_lslong": 10794658036228347919}, "enable": true, "created": "2025-09-01T01:35:49.567966", "last_modified": "2025-09-01T01:35:49.567966", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"virtual_router_dpdk_enabled : falsevirtual_router_ip_address : "10.0.0.22" } deleted = False >> 09/01/2025 01:35:49.576 7fbb3e0270e0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.584 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:35:49.585 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field access_control_lists in resource virtual-network: 'access_control_lists' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_pools in resource virtual-network: 'floating_ip_pools' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_pools in resource virtual-network: 'alias_ip_pools' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instances in resource virtual-network: 'routing_instances' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domains in resource virtual-network: 'bridge_domains' 09/01/2025 01:35:49.643 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-network: 'security_logging_object_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-network: 'qos_config_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-network: 'network_ipam_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_refs in resource virtual-network: 'virtual_network_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-network: 'service_health_check_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_table_refs in resource virtual-network: 'route_table_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field multicast_policy_refs in resource virtual-network: 'multicast_policy_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgpvpn_refs in resource virtual-network: 'bgpvpn_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_map_refs in resource virtual-network: 'intent_map_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_refs in resource virtual-network: 'routing_policy_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-network: 'tag_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_network_back_refs in resource virtual-network: 'virtual_network_back_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-network: 'virtual_machine_interface_back_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-network: 'instance_ip_back_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-network: 'physical_router_back_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_back_refs in resource virtual-network: 'port_tuple_back_refs' 09/01/2025 01:35:49.644 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-network: 'logical_router_back_refs' 09/01/2025 01:35:49.645 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_node_back_refs in resource virtual-network: 'flow_node_back_refs' 09/01/2025 01:35:49.645 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field firewall_rule_back_refs in resource virtual-network: 'firewall_rule_back_refs' 09/01/2025 01:35:49.645 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnect_back_refs in resource virtual-network: 'data_center_interconnect_back_refs' 09/01/2025 01:35:49.645 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabric_back_refs in resource virtual-network: 'fabric_back_refs' 09/01/2025 01:35:49.645 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field host_based_service_back_refs in resource virtual-network: 'host_based_service_back_refs' 09/01/2025 01:35:49.654 7fbb3ceb7540 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Object with fq_name ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'vhost0'] not found in DB: Unknown id: virtual_machine_interface default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 09/01/2025 01:35:49.657 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << object_type = virtual_machine_interface identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 url = http://10.0.0.50:8082/virtual-machine-interfaces operation = post useragent = an-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.22 body = {"virtual-machine-interface": {"uuid": null, "fq_name": ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1", "vhost0"], "parent_type": "virtual-router", "virtual_machine_interface_disable_policy": false, "virtual_network_refs": [{"to": ["default-domain", "default-project", "ip-fabric"], "uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe"}], "id_perms": {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": null, "enable": true, "created": null, "last_modified": null, "description": null, "user_visible": true, "creator": null}, "perms2": {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}, "display_name": "vhost0"}} domain = default-domain >> 09/01/2025 01:35:49.680 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'vhost0'], 'obj_dict': {'display_name': 'vhost0', 'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'vhost0'], 'id_perms': {'created': '2025-09-01T01:35:49.668311', 'creator': None, 'description': None, 'enable': True, 'last_modified': '2025-09-01T01:35:49.668311', 'permissions': {'group': 'cloud-admin-group', 'group_access': 7, 'other_access': 7, 'owner': 'cloud-admin', 'owner_access': 7}, 'user_visible': True, 'uuid': {'uuid_lslong': 12472940248073749293, 'uuid_mslong': 15418220656626518203}}, 'parent_type': 'virtual-router', 'parent_uuid': 'a8ce144b-9496-4f0b-95ce-544a9934d80f', 'perms2': {'global_access': 0, 'owner': 'cloud-admin', 'owner_access': 7, 'share': []}, 'uuid': 'd5f885ff-7ad0-48bb-ad18-c912371bc32d', 'virtual_machine_interface_disable_policy': False, 'virtual_machine_interface_mac_addresses': {'mac_address': ['02:d5:f8:85:ff:7a']}, 'virtual_network_refs': [{'to': ['default-domain', 'default-project', 'ip-fabric'], 'uuid': 'e07e6e04-4e93-4043-ab83-32b8f8854bbe'}]}, 'oper': 'CREATE', 'request-id': 'req-f6bff454-c5c2-4c20-b305-c5e32915e30b', 'type': 'virtual_machine_interface', 'uuid': 'd5f885ff-7ad0-48bb-ad18-c912371bc32d'} 09/01/2025 01:35:49.680 7fbb3decd5e0 [contrail-api] [DEBUG]: Add uve in the [ObjectVMITable:ContrailConfigTrace] map 09/01/2025 01:35:49.681 7fbb3cf35b80 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.681 7fbb3cf35b80 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1", "vhost0"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15418220656626518203, "uuid_lslong": 12472940248073749293}, "enable": true, "created": "2025-09-01T01:35:49.668311", "last_modified": "2025-09-01T01:35:49.668311", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "d5f885ff-7ad0-48bb-ad18-c912371bc32d"virtual_machine_interface_disable_policy : falsevirtual_machine_interface_mac_addresses : {"mac_address": ["02:d5:f8:85:ff:7a"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe"}] } deleted = False >> 09/01/2025 01:35:49.682 7fbb3cf35b80 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.687 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d5f885ff-7ad0-48bb-ad18-c912371bc32d object_type = virtual_machine_interface identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 url = http://127.0.0.1/ref-update operation = ref-update domain = default-domain >> 09/01/2025 01:35:49.704 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-f6bff454-c5c2-4c20-b305-c5e32915e30b', 'type': 'virtual_machine_interface', 'uuid': 'd5f885ff-7ad0-48bb-ad18-c912371bc32d'} 09/01/2025 01:35:49.707 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.708 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1", "vhost0"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15418220656626518203, "uuid_lslong": 12472940248073749293}, "enable": true, "created": "2025-09-01T01:35:49.668311", "last_modified": "2025-09-01T01:35:49.696689", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}routing_instance_refs : [{"to": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "23054292-38bc-4b6f-b5fa-0f8021a7362f"}]uuid : "d5f885ff-7ad0-48bb-ad18-c912371bc32d"virtual_machine_interface_disable_policy : falsevirtual_machine_interface_mac_addresses : {"mac_address": ["02:d5:f8:85:ff:7a"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "attr": null, "uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe"}] } deleted = False >> 09/01/2025 01:35:49.708 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:49.741 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:50.743 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:51.172 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:35:51.173 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:35:51.744 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:52.745 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:53.210 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:53.212 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:53.213 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:53.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:54.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:55.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:56.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:57.213 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:35:57.214 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:57.215 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:35:57.215 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:57.215 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:57.215 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:35:57.215 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:35:57.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:58.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:35:59.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:00.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:01.214 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:01.216 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:01.217 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:01.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:02.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:03.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:04.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:05.217 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:05.218 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:05.219 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:05.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:06.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:07.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:08.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:09.219 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:09.220 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:09.221 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:09.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:10.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:11.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:12.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:13.221 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:13.222 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:13.223 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:13.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:14.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:15.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:16.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:17.222 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:17.223 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:17.224 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:17.225 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:18.741 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:19.742 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:20.744 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:21.225 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:21.226 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:21.227 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:21.745 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:22.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:23.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:24.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:25.227 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:25.228 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:25.229 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:25.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:26.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:27.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:28.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:29.229 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:29.230 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:29.230 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:29.230 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:29.230 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:29.230 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:29.231 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:29.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:30.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:31.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:32.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:33.231 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:33.232 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:33.232 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:33.232 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:33.232 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:33.232 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:33.233 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:33.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:34.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:35.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:36.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:37.233 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:37.234 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:37.235 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:37.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:38.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:39.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:40.668 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:36:40.671 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:36:40.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:41.235 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:41.236 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:41.236 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:41.236 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:41.237 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:41.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:42.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:43.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:44.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:45.237 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:45.238 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:45.239 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:45.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:46.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:48.743 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:49.239 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:49.240 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:49.241 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:49.745 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:50.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:51.193 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:36:51.193 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:36:51.193 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:36:51.193 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:36:51.193 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:36:51.194 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:36:51.194 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:36:51.194 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:36:51.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:52.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:53.240 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:53.241 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:53.241 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:53.242 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:53.243 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:53.243 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:53.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:54.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:55.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:56.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:57.242 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:57.244 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:36:57.245 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:36:57.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:58.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:36:59.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:00.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:01.244 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:01.245 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:01.246 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:01.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:02.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:03.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:04.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:05.246 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:05.247 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:05.248 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:05.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:06.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:07.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:08.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:09.248 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:09.249 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:09.249 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:09.250 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:09.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:10.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:11.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:12.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:13.249 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:13.250 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:13.251 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:13.252 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:13.252 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:13.252 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:13.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:14.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:15.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:16.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:17.251 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:17.253 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:17.254 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:17.254 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:17.254 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:17.254 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:18.744 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:19.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:20.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:21.254 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:21.255 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:21.256 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:21.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:22.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:23.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:24.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:25.256 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:25.257 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:25.258 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:25.258 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:25.258 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:25.258 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:25.258 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:25.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:26.246 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Parent key not found for resource global-system-config: 'parent_type' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field feature_flags in resource global-system-config: 'feature_flags' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field control_node_zones in resource global-system-config: 'control_node_zones' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_vrouter_configs in resource global-system-config: 'global_vrouter_configs' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_qos_configs in resource global-system-config: 'global_qos_configs' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_routers in resource global-system-config: 'virtual_routers' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_nodes in resource global-system-config: 'config_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_nodes in resource global-system-config: 'analytics_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field flow_nodes in resource global-system-config: 'flow_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field devicemgr_nodes in resource global-system-config: 'devicemgr_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field database_nodes in resource global-system-config: 'database_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field webui_nodes in resource global-system-config: 'webui_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_database_nodes in resource global-system-config: 'config_database_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_alarm_nodes in resource global-system-config: 'analytics_alarm_nodes' 09/01/2025 01:37:26.247 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field analytics_snmp_nodes in resource global-system-config: 'analytics_snmp_nodes' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_appliance_sets in resource global-system-config: 'service_appliance_sets' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field api_access_lists in resource global-system-config: 'api_access_lists' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alarms in resource global-system-config: 'alarms' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field config_propertiess in resource global-system-config: 'config_propertiess' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field job_templates in resource global-system-config: 'job_templates' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field data_center_interconnects in resource global-system-config: 'data_center_interconnects' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field intent_maps in resource global-system-config: 'intent_maps' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field fabrics in resource global-system-config: 'fabrics' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field node_profiles in resource global-system-config: 'node_profiles' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_routers in resource global-system-config: 'physical_routers' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field device_images in resource global-system-config: 'device_images' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field nodes in resource global-system-config: 'nodes' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field features in resource global-system-config: 'features' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_roles in resource global-system-config: 'physical_roles' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field overlay_roles in resource global-system-config: 'overlay_roles' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field role_definitions in resource global-system-config: 'role_definitions' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field global_analytics_configs in resource global-system-config: 'global_analytics_configs' 09/01/2025 01:37:26.248 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource global-system-config: 'tag_refs' 09/01/2025 01:37:26.249 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_back_refs in resource global-system-config: 'qos_config_back_refs' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_routers in resource routing-instance: 'bgp_routers' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_refs in resource routing-instance: 'routing_instance_refs' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_target_refs in resource routing-instance: 'route_target_refs' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource routing-instance: 'tag_refs' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource routing-instance: 'virtual_machine_interface_back_refs' 09/01/2025 01:37:26.256 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field route_aggregate_back_refs in resource routing-instance: 'route_aggregate_back_refs' 09/01/2025 01:37:26.257 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_policy_back_refs in resource routing-instance: 'routing_policy_back_refs' 09/01/2025 01:37:26.257 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field routing_instance_back_refs in resource routing-instance: 'routing_instance_back_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:37:26.266 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:37:26.269 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a8ce144b-9496-4f0b-95ce-544a9934d80f object_type = virtual_router identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/virtual-router/a8ce144b-9496-4f0b-95ce-544a9934d80f operation = http_put useragent = an-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.22 body = {"virtual-router": {"uuid": "a8ce144b-9496-4f0b-95ce-544a9934d80f", "virtual_router_dpdk_enabled": false}} domain = default-domain >> 09/01/2025 01:37:26.278 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1'], 'oper': 'UPDATE', 'request-id': 'req-289c98ce-ddb2-46fa-a5f5-966a1b563ad8', 'type': 'virtual_router', 'uuid': 'a8ce144b-9496-4f0b-95ce-544a9934d80f'} 09/01/2025 01:37:26.281 7fbb3e027e00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:26.281 7fbb3e027e00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = an-jenkins-deploy-platform-juju-k8s-1454-1 elements = { display_name : "an-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12163681958422269707, "uuid_lslong": 10794658036228347919}, "enable": true, "created": "2025-09-01T01:35:49.567966", "last_modified": "2025-09-01T01:37:26.272555", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"virtual_router_dpdk_enabled : falsevirtual_router_ip_address : "10.0.0.22" } deleted = False >> 09/01/2025 01:37:26.281 7fbb3e027e00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:26.325 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:37:26.326 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:37:26.333 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_logging_object_refs in resource virtual-machine-interface: 'security_logging_object_refs' 09/01/2025 01:37:26.333 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field qos_config_refs in resource virtual-machine-interface: 'qos_config_refs' 09/01/2025 01:37:26.333 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field security_group_refs in resource virtual-machine-interface: 'security_group_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_refs in resource virtual-machine-interface: 'virtual_machine_interface_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-machine-interface: 'virtual_machine_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_router_refs in resource virtual-machine-interface: 'bgp_router_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_tuple_refs in resource virtual-machine-interface: 'port_tuple_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_health_check_refs in resource virtual-machine-interface: 'service_health_check_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field interface_route_table_refs in resource virtual-machine-interface: 'interface_route_table_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_interface_refs in resource virtual-machine-interface: 'physical_interface_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bridge_domain_refs in resource virtual-machine-interface: 'bridge_domain_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field service_endpoint_refs in resource virtual-machine-interface: 'service_endpoint_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_refs in resource virtual-machine-interface: 'virtual_port_group_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field port_profile_refs in resource virtual-machine-interface: 'port_profile_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-machine-interface: 'tag_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interface_back_refs in resource virtual-machine-interface: 'virtual_machine_interface_back_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-machine-interface: 'instance_ip_back_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field subnet_back_refs in resource virtual-machine-interface: 'subnet_back_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field floating_ip_back_refs in resource virtual-machine-interface: 'floating_ip_back_refs' 09/01/2025 01:37:26.334 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field alias_ip_back_refs in resource virtual-machine-interface: 'alias_ip_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_interface_back_refs in resource virtual-machine-interface: 'logical_interface_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field bgp_as_a_service_back_refs in resource virtual-machine-interface: 'bgp_as_a_service_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field customer_attachment_back_refs in resource virtual-machine-interface: 'customer_attachment_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field logical_router_back_refs in resource virtual-machine-interface: 'logical_router_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_pool_back_refs in resource virtual-machine-interface: 'loadbalancer_pool_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_ip_back_refs in resource virtual-machine-interface: 'virtual_ip_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field loadbalancer_back_refs in resource virtual-machine-interface: 'loadbalancer_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_port_group_back_refs in resource virtual-machine-interface: 'virtual_port_group_back_refs' 09/01/2025 01:37:26.335 7fbb3df45ae0 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field link_aggregation_group_back_refs in resource virtual-machine-interface: 'link_aggregation_group_back_refs' 09/01/2025 01:37:26.338 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = d5f885ff-7ad0-48bb-ad18-c912371bc32d object_type = virtual_machine_interface identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 url = http://10.0.0.50:8082/virtual-machine-interface/d5f885ff-7ad0-48bb-ad18-c912371bc32d operation = http_put useragent = an-jenkins-deploy-platform-juju-k8s-1454-1:/opt/contrail/utils/provision_vrouter.py remote_ip = 10.0.0.22 body = {"virtual-machine-interface": {"uuid": "d5f885ff-7ad0-48bb-ad18-c912371bc32d", "virtual_machine_interface_disable_policy": false}} domain = default-domain >> 09/01/2025 01:37:26.344 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1', 'vhost0'], 'oper': 'UPDATE', 'request-id': 'req-b7f2219d-1994-43ae-91a0-48b4f5976fda', 'type': 'virtual_machine_interface', 'uuid': 'd5f885ff-7ad0-48bb-ad18-c912371bc32d'} 09/01/2025 01:37:26.355 7fbb3e027e00 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:26.355 7fbb3e027e00 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1:vhost0 elements = { display_name : "vhost0"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1", "vhost0"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 15418220656626518203, "uuid_lslong": 12472940248073749293}, "enable": true, "created": "2025-09-01T01:35:49.668311", "last_modified": "2025-09-01T01:37:26.341561", "description": null, "user_visible": true, "creator": null}parent_type : "virtual-router"parent_uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}routing_instance_refs : [{"to": ["default-domain", "default-project", "ip-fabric", "ip-fabric"], "attr": {"direction": "both", "vlan_tag": null, "src_mac": null, "dst_mac": null, "mpls_label": null, "service_chain_address": null, "ipv6_service_chain_address": null, "protocol": null}, "uuid": "23054292-38bc-4b6f-b5fa-0f8021a7362f"}]uuid : "d5f885ff-7ad0-48bb-ad18-c912371bc32d"virtual_machine_interface_disable_policy : falsevirtual_machine_interface_mac_addresses : {"mac_address": ["02:d5:f8:85:ff:7a"]}virtual_network_refs : [{"to": ["default-domain", "default-project", "ip-fabric"], "attr": null, "uuid": "e07e6e04-4e93-4043-ab83-32b8f8854bbe"}] } deleted = False >> 09/01/2025 01:37:26.356 7fbb3e027e00 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:26.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:27.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:28.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:29.258 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:29.259 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:29.260 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:29.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:30.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:31.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:32.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:33.260 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:33.261 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:33.261 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:33.261 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:33.262 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:33.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:34.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:35.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:36.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:37.262 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:37.263 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:37.264 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:37.265 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:37.265 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:37.265 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:37.265 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:37.265 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:37.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:38.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:39.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:40.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:41.264 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:41.265 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:41.265 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:41.265 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:41.265 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:41.266 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:41.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:42.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:43.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:44.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:45.266 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:45.267 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:45.268 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:45.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:46.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:48.745 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:49.268 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:49.269 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:49.270 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:49.270 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:49.270 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:49.270 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:49.270 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:49.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:50.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:37:51.218 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:37:51.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:52.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:53.270 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:53.271 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:53.271 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:53.272 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:53.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:54.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:55.521 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:37:55.523 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:37:55.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:56.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:57.272 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:37:57.273 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:37:57.273 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:37:57.274 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:37:57.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:57.880 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:37:57.882 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:37:58.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:37:59.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:00.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:01.274 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:01.275 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:01.276 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:01.382 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:38:01.385 7fbb3ceb7540 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:38:01.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:02.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:03.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:04.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:05.276 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:05.277 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:05.278 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:05.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:06.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:07.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:08.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:09.278 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:09.279 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:09.280 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:09.280 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:09.280 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:09.280 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:09.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:10.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:11.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:12.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:13.280 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:13.281 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:13.282 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:13.283 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:13.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:14.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:15.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:16.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:17.282 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:17.283 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:17.283 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:17.283 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:17.283 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:17.284 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:18.746 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:19.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:20.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:21.285 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:21.286 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:21.287 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:21.287 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:21.287 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:21.287 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:21.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:22.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:23.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:24.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:25.286 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:25.287 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:25.288 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:25.289 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:25.289 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:25.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:26.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:27.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:28.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:29.289 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:29.290 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:29.291 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:29.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:30.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:31.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:32.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:33.291 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:33.292 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:33.292 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:33.292 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:33.293 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:33.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:34.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:35.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:36.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:37.293 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:37.294 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:37.295 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:37.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:38.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:39.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:40.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:41.295 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:41.296 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:41.297 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:41.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:42.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:43.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:44.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:45.298 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:45.299 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:45.300 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:45.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:46.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:48.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:49.300 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:49.301 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:49.302 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:49.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:50.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:38:51.236 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:38:51.237 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:38:51.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:52.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:53.302 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:53.303 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:53.304 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:53.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:54.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:55.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:56.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:57.305 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:38:57.306 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:38:57.307 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:38:57.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:58.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:38:59.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:00.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:01.306 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:01.307 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:01.307 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:01.307 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:01.308 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:01.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:02.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:03.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:04.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:05.308 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:05.309 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:05.309 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:05.309 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:05.310 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:05.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:06.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:07.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:08.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:09.310 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:09.329 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:09.330 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:09.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:10.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:11.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:12.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:13.329 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:13.331 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:13.332 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:13.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:14.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:15.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:16.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:17.332 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:17.333 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:17.333 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:17.333 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:17.334 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:18.747 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:19.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:20.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:21.335 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:21.336 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:21.337 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:21.337 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:21.337 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:21.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:22.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:23.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:24.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:25.336 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:25.337 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:25.337 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:25.338 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:25.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:26.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:27.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:28.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:29.338 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:29.339 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:29.339 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:29.339 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:29.339 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:29.340 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:29.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:30.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:31.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:32.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:33.340 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:33.341 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:33.342 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:33.343 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:33.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:34.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:35.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:36.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:37.342 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:37.343 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:37.343 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:37.344 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:37.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:38.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:39.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:40.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:41.344 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:41.345 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:41.346 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:41.347 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:41.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:42.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:43.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:44.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:45.347 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:45.348 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:45.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:46.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:48.748 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:49.348 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:49.349 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:49.349 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:49.349 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:49.349 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:49.349 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:49.350 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:49.749 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:50.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:39:51.259 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:39:51.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:52.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:53.350 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:53.352 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:53.353 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:53.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:54.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:55.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:56.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:57.352 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:39:57.353 7fbb3ceb7540 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:57.354 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:39:57.355 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:57.355 7fbb3ceb7540 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:57.355 7fbb3ceb7540 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:39:57.355 7fbb3ceb7540 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:57.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:58.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:59.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_refs in resource virtual-router: 'virtual_machine_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:39:59.794 7fbb3ceb7400 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:39:59.798 7fbb3ceb7400 [contrail-api] [ERROR]: SANDESH: [DROP: NoSession] __default__ [SYS_INFO]: VncApiConfigLog: api_log = << identifier_uuid = a8ce144b-9496-4f0b-95ce-544a9934d80f object_type = virtual_router identifier_name = default-global-system-config:an-jenkins-deploy-platform-juju-k8s-1454-1 url = http://10.0.0.50:8082/ref-update operation = ref-update useragent = cn-jenkins-deploy-platform-juju-k8s-1454-1:/usr/local/bin/contrail-kube-manager remote_ip = 10.0.0.50 body = {"type": "virtual-router", "uuid": "a8ce144b-9496-4f0b-95ce-544a9934d80f", "ref-type": "virtual-machine", "ref-uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8", "ref-fq-name": null, "operation": "ADD", "attr": null} domain = default-domain >> 09/01/2025 01:39:59.804 7fbb3decd5e0 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: Notification Message: {'fq_name': ['default-global-system-config', 'an-jenkins-deploy-platform-juju-k8s-1454-1'], 'oper': 'UPDATE', 'request-id': 'req-10dc3d7c-4558-47f2-a814-0dc448db550a', 'type': 'virtual_router', 'uuid': 'a8ce144b-9496-4f0b-95ce-544a9934d80f'} 09/01/2025 01:39:59.812 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:39:59.813 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] ContrailConfigTrace: data = << name = an-jenkins-deploy-platform-juju-k8s-1454-1 elements = { display_name : "an-jenkins-deploy-platform-juju-k8s-1454-1"fq_name : ["default-global-system-config", "an-jenkins-deploy-platform-juju-k8s-1454-1"]id_perms : {"permissions": {"owner": "cloud-admin", "owner_access": 7, "group": "cloud-admin-group", "group_access": 7, "other_access": 7}, "uuid": {"uuid_mslong": 12163681958422269707, "uuid_lslong": 10794658036228347919}, "enable": true, "created": "2025-09-01T01:35:49.567966", "last_modified": "2025-09-01T01:39:59.800989", "description": null, "user_visible": true, "creator": null}parent_type : "global-system-config"parent_uuid : "70eaa5ad-608f-40c9-a193-2be391842efd"perms2 : {"owner": "cloud-admin", "owner_access": 7, "global_access": 0, "share": []}uuid : "a8ce144b-9496-4f0b-95ce-544a9934d80f"virtual_machine_refs : [{"to": ["k8s__kube-system__coredns-6f867cd986-wlkqx"], "attr": null, "uuid": "a4a425f1-8708-4c1c-b41f-cd30b9d1d8a8"}]virtual_router_dpdk_enabled : falsevirtual_router_ip_address : "10.0.0.22" } deleted = False >> 09/01/2025 01:39:59.813 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:00.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:01.354 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:01.355 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:01.355 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:01.356 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:01.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:02.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:03.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:04.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:05.357 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:05.358 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:05.359 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:05.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:06.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:07.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:08.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:09.358 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:09.360 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:09.361 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:09.361 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:09.361 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:09.361 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:09.361 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:09.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:10.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:11.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:12.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:13.360 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:13.361 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:13.362 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:13.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:14.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:15.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:16.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:17.362 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:17.363 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:17.363 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:17.363 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:17.363 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:17.363 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:17.364 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:18.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:19.750 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:20.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:21.365 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:21.366 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:21.367 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:21.367 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:21.367 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:21.752 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:22.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:23.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:24.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:25.366 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:25.367 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:25.367 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:25.368 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:25.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:26.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:27.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:28.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:29.369 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:29.370 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:29.370 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:29.370 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:29.370 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:29.370 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:29.371 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:29.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:30.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:31.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:32.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:33.371 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:33.372 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:33.372 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:33.372 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:33.372 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:33.372 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:33.373 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:33.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:34.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:35.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:36.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:37.373 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:37.374 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:37.375 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:37.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:38.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:39.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:40.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:41.374 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:41.376 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:41.377 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:41.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:42.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:43.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:44.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:45.377 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:45.378 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:45.379 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:45.380 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:45.380 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:45.380 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:45.380 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:45.380 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:45.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:46.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:48.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:49.380 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:49.381 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:49.382 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:49.382 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:49.382 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:49.382 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:49.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:49.925 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:40:49.928 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:40:50.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:51.327 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:40:51.328 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:40:51.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:52.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:53.382 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:53.383 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:53.384 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:53.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:54.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:55.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:56.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:57.384 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:40:57.385 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:40:57.386 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:40:57.387 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:40:57.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:58.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:40:59.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:00.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:01.387 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:01.388 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:01.389 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:01.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:02.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:03.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:04.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:05.388 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:05.389 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:05.389 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:05.389 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:05.389 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:05.390 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:05.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:06.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:07.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:08.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:09.390 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:09.392 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:09.393 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:09.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:10.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:11.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:12.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:13.393 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:13.394 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:13.395 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:13.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:14.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:15.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:16.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:17.394 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:17.395 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:17.396 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:18.751 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:19.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:20.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:21.396 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:21.397 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:21.397 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:21.397 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:21.397 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:21.398 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:21.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:22.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:23.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:24.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:25.398 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:25.399 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:25.399 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:25.399 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:25.400 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:25.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:26.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:27.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:28.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:29.401 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:29.402 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:29.403 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:29.403 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:29.403 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:29.403 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:29.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:30.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:31.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:32.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:33.403 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:33.404 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:33.405 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:33.405 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:33.405 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:33.405 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:33.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:34.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:35.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:36.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:37.405 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:37.406 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:37.407 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:37.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:38.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:39.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:40.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:41.407 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:41.408 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:41.408 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:41.409 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:41.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:42.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:43.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:44.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:45.409 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:45.410 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:45.410 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:45.410 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:45.411 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:45.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:46.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:48.753 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:49.411 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:49.412 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:49.413 7fbb3df45ae0 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:49.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:50.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:41:51.350 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:41:51.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:52.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:53.413 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:53.414 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:53.414 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:53.414 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:53.414 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:53.415 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:53.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:54.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:55.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:56.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:57.415 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:41:57.416 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:41:57.417 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:41:57.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:58.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:41:59.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:00.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:01.417 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:01.418 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:01.419 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:01.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:02.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:03.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:04.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:05.419 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:05.419 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:05.420 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:05.421 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:05.421 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:05.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:06.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:07.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:08.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:09.421 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:09.422 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:09.423 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:09.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:10.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:11.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:12.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:13.423 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:13.424 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:13.424 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:13.424 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:13.424 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:13.424 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:13.425 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:13.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:14.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:15.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:16.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:17.425 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:17.426 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:17.426 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:17.426 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:17.427 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:17.428 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:18.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:19.755 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:20.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:21.427 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:21.428 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:21.429 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:21.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:22.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:23.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:24.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:25.430 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:25.431 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:25.432 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:25.432 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:25.432 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:25.432 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:25.432 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:25.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:26.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:27.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:28.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:29.432 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:29.433 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:29.434 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:29.434 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:29.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:30.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:31.738 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:42:31.741 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:42:31.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:32.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:33.434 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:33.435 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:33.436 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:33.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:34.092 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:42:34.095 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:42:34.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:35.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:36.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:37.436 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:37.437 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:37.438 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:37.695 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:42:37.698 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:42:37.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:38.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:39.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:40.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:41.438 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:41.439 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:41.440 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:41.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:42.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:43.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:44.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:45.439 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:45.440 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:45.440 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:45.440 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:45.440 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:45.440 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:45.441 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:45.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:46.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:48.754 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:49.441 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:49.442 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:49.443 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:49.443 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:49.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:50.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:42:51.372 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:42:51.373 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:42:51.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:52.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:53.443 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:53.444 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:53.445 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:53.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:54.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:55.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:56.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:57.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:42:57.446 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:42:57.447 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:42:57.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:58.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:42:59.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:00.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:01.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:01.448 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:01.448 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:01.448 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:01.449 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:01.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:02.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:03.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:04.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:05.450 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:05.451 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:05.452 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:05.452 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:05.452 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:05.452 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:05.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:06.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:07.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:08.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:09.452 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:09.453 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:09.454 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:09.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:10.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:11.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:12.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:13.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:13.455 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:13.456 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:13.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:14.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:15.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:16.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:17.456 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:17.457 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:17.458 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:18.756 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:19.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:20.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:21.458 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:21.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:21.460 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:21.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:22.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:23.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:24.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:25.460 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:25.461 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:25.462 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:25.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:26.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:27.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:28.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:29.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:29.464 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:29.465 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:29.465 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:29.465 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:29.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:30.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:31.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:32.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:33.465 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:33.466 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:33.467 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:33.467 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:33.467 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:33.467 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:33.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:34.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:35.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:36.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:37.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:37.467 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:37.468 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:37.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:38.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:39.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:40.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:41.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:41.469 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:41.470 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:41.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:42.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:43.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:44.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:45.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:45.471 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:45.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:45.473 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:45.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:46.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:48.757 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:49.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:49.473 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:49.473 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:49.473 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:49.473 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:49.474 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:49.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:50.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:43:51.437 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:43:51.438 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:43:51.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:52.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:53.474 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:53.475 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:53.476 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:53.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:54.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:55.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:56.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:57.477 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:43:57.478 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:43:57.478 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:43:57.478 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:57.478 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:43:57.479 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:43:57.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:58.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:43:59.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:00.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:01.478 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:01.480 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:01.481 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:01.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:02.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:03.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:04.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:05.481 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:05.482 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:05.482 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:05.482 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:05.483 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:05.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:06.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:07.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:08.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:09.482 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:09.483 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:09.483 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:09.484 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:09.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:10.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:11.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:12.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:13.484 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:13.485 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:13.486 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:13.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:14.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:15.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:16.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:17.486 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:17.488 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:17.489 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:17.489 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:17.489 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:17.489 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:18.758 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:19.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:20.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:21.489 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:21.490 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:21.490 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:21.490 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:21.491 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:21.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:22.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:23.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:24.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:25.491 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:25.492 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:25.492 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:25.492 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:25.492 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:25.493 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:25.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:26.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:27.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:28.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:29.494 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:29.495 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:29.496 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:29.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:30.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:31.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:32.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:33.496 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:33.497 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:33.498 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:33.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:34.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:35.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:36.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:37.498 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:37.499 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:37.500 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:37.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:38.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:39.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:40.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:41.500 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:41.501 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:41.502 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:41.503 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:41.503 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:41.503 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:41.503 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:41.503 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:41.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:42.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:43.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:44.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:45.503 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:45.504 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:45.505 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:45.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:46.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:48.759 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:49.505 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:49.506 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:49.506 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:49.506 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:49.506 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:49.507 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:49.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:50.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:44:51.460 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:44:51.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:52.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:53.508 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:53.508 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:53.509 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:53.510 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:53.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:54.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:55.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:56.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:57.510 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:44:57.511 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:44:57.512 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:44:57.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:58.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:44:59.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:00.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:01.512 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:01.513 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:01.513 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:01.513 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:01.513 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:01.514 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:01.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:02.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:03.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:04.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:05.515 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:05.516 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:05.517 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:05.517 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:05.517 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:05.517 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:05.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:06.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:07.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:08.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:09.517 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:09.518 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:09.518 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:09.518 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:09.518 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:09.518 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:09.519 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:09.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:10.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:11.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:12.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:13.519 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:13.520 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:13.521 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:13.521 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:13.521 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:13.521 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:13.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:14.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:15.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:16.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:17.521 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:17.522 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:17.523 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:18.760 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:19.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:20.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:21.523 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:21.524 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:21.524 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:21.525 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:21.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:22.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:23.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:24.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:25.526 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:25.527 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:25.528 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:25.528 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:25.528 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:25.528 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:25.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:26.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:27.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:28.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:29.528 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:29.529 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:29.529 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:29.529 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:29.530 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:29.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:30.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:31.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:32.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:33.530 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:33.531 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:33.531 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:33.531 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:33.531 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:33.531 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:33.532 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:33.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:34.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:35.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:36.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:37.532 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:37.533 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:37.533 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:37.533 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:37.533 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:37.533 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:37.534 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:37.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:38.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:39.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:40.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:41.534 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:41.535 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:41.535 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:41.535 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:41.536 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:41.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:42.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:43.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:44.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:45.537 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:45.538 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:45.539 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:45.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:46.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:48.761 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:49.539 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:49.540 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:49.540 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:49.540 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:49.540 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:49.540 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:49.541 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:49.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:50.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:51.482 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:45:51.483 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:45:51.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:52.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:53.541 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:53.542 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:53.543 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:53.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:54.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:55.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:56.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:57.544 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:57.545 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:45:57.546 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:45:57.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:58.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:45:59.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:00.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:01.545 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:01.546 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:01.547 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:01.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:02.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:03.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:04.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:05.547 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:05.548 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:05.548 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:05.548 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:05.548 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:05.549 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:05.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:06.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:07.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:08.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:09.550 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:09.551 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:09.552 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:09.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:10.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:11.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:12.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:13.552 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:13.553 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:13.554 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:13.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:14.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:15.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:16.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:17.554 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:17.555 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:17.556 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:17.556 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:17.556 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:17.556 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:17.556 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:18.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:19.763 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:20.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:21.556 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:21.557 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:21.558 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:21.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:22.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:23.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:24.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:25.558 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:25.559 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:25.560 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:25.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:26.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:27.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:28.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:29.560 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:29.561 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:29.562 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:29.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:30.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:31.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:32.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:33.562 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:33.563 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:33.563 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:33.564 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:33.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:34.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:35.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:36.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:37.564 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:37.565 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:37.566 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:37.566 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:37.566 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:37.566 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:37.566 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:37.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:38.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:39.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:40.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:41.565 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:41.566 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:41.567 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:41.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:42.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:43.631 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:46:43.634 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:46:43.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:44.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:45.567 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:45.568 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:45.568 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:45.568 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:45.569 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:45.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:46.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:48.762 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:49.569 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:49.570 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:49.570 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:49.570 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:49.570 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:49.571 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:49.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:50.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:46:51.504 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:46:51.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:52.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:53.571 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:53.572 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:53.572 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:53.573 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:53.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:54.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:55.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:56.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:57.573 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:46:57.588 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:46:57.589 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:46:57.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:58.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:46:59.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:00.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:01.589 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:01.590 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:01.591 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:01.591 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:01.591 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:01.591 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:01.591 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:01.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:02.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:03.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:04.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:05.591 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:05.592 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:05.592 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:05.592 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:05.593 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:05.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:06.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:07.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:08.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:09.593 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:09.594 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:09.594 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:09.595 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:09.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:10.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:11.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:12.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:13.595 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:13.596 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:13.597 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:13.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:14.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:15.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:16.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:17.598 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:17.599 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:17.600 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:17.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:17.600 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:17.600 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:18.764 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:19.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:20.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:21.600 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:21.601 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:21.602 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:21.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:22.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:23.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:24.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:25.602 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:25.603 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:25.603 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:25.603 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:25.604 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:25.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:26.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:27.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:28.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:29.604 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:29.605 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:29.605 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:29.605 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:29.606 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:29.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:30.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:31.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:32.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:33.607 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:33.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:33.609 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:33.609 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:33.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:33.609 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:33.609 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:33.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:34.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:35.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:36.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:37.608 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:37.609 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:37.610 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:37.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:38.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:39.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:40.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:41.610 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:41.611 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:41.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:42.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:43.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:44.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:45.612 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:45.613 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:45.614 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:45.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:46.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:48.765 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:49.614 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:49.615 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:49.615 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:49.615 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:49.616 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:49.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:50.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:47:51.526 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:47:51.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:52.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:53.617 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:53.618 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:53.619 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:53.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:54.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:55.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:56.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:57.619 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:47:57.620 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:47:57.621 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:57.622 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:47:57.622 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:57.622 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:57.622 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:47:57.622 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:47:57.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:58.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:47:59.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:00.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:01.621 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:01.622 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:01.623 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:01.624 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:01.624 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:01.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:02.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:03.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:04.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:05.623 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:05.624 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:05.625 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:05.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:06.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:07.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:08.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:09.626 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:09.627 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:09.628 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:09.628 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:09.628 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:09.628 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:09.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:10.658 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:48:10.660 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:48:10.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:11.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:12.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:13.440 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:48:13.442 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:48:13.628 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:13.629 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:13.630 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:13.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:14.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:15.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:16.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:16.992 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:48:16.996 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:48:17.630 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:17.631 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:17.631 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:17.631 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:17.631 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:17.632 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:18.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:19.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:20.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:21.633 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:21.634 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:21.634 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:21.634 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:21.634 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:21.635 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:21.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:22.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:23.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:24.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:25.635 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:25.636 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:25.637 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:25.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:26.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:27.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:28.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:29.638 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:29.639 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:29.640 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:29.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:30.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:31.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:32.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:33.640 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:33.640 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:33.641 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:33.642 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:33.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:34.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:35.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:36.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:37.642 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:37.643 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:37.644 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:37.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:38.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:39.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:40.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:41.644 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:41.646 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:41.647 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:41.647 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:41.647 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:41.647 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:41.647 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:41.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:42.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:43.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:44.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:45.647 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:45.648 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:45.649 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:45.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:46.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:48.766 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:49.649 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:49.650 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:49.651 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:49.767 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:50.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:51.594 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:48:51.594 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:48:51.594 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:48:51.595 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:48:51.595 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:48:51.595 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:48:51.595 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:48:51.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:52.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:53.652 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:53.653 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:53.654 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:53.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:54.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:55.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:56.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:57.653 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:48:57.654 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:48:57.655 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:48:57.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:58.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:48:59.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:00.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:01.655 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:01.656 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:01.656 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:01.657 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:01.658 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:01.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:02.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:03.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:04.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:05.658 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:05.659 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:05.659 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:05.659 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:05.659 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:05.660 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:05.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:06.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:07.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:08.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:09.661 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:09.662 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:09.663 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:09.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:10.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:11.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:12.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:13.663 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:13.664 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:13.665 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:13.665 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:13.665 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:13.665 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:13.665 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:13.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:14.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:15.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:16.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:17.666 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:17.667 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:17.668 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:17.668 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:17.668 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:17.668 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:17.668 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:18.768 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:19.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:20.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:21.668 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:21.669 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:21.669 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:21.669 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:21.670 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:21.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:22.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:23.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:24.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:25.670 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:25.671 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:25.672 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:25.673 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:25.673 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:25.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:26.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:27.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:28.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:29.673 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:29.674 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:29.674 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:29.674 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:29.675 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:29.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:30.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:31.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:32.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:33.676 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:33.677 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:33.678 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:33.678 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:33.678 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:33.678 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:33.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:34.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:35.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:36.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:37.678 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:37.679 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:37.680 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:37.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:38.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:39.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:40.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:41.679 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:41.681 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:41.682 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:41.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:42.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:43.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:44.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:45.682 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:45.683 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:45.684 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:45.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:46.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:48.769 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:49.684 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:49.686 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:49.687 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:49.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:50.771 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:49:51.617 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:49:51.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:52.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:53.687 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:53.688 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:53.689 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:53.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:54.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:55.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:56.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:57.690 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:49:57.691 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:57.692 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:49:57.692 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:57.692 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:57.692 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:49:57.692 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:49:57.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:58.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:49:59.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:00.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:01.692 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:01.693 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:01.693 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:01.693 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:01.694 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:01.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:02.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:03.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:04.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:05.694 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:05.695 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:05.695 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:05.695 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:05.696 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:05.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:06.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:07.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:08.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:09.697 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:09.698 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:09.699 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:09.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:10.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:11.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:12.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:13.699 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:13.700 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:13.701 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:13.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:14.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:15.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:16.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:17.702 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:17.703 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:17.704 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:17.704 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:17.704 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:17.704 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:17.704 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:18.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:19.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:20.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:21.704 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:21.705 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:21.706 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:21.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:22.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:23.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:24.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:25.706 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:25.707 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:25.708 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:25.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:26.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:27.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:28.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:29.709 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:29.710 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:29.711 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:29.711 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:29.711 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:29.711 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:29.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:30.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:31.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:32.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:33.711 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:33.712 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:33.712 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:33.712 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:33.713 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:33.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:34.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:35.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:36.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:37.713 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:37.714 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:37.715 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:37.716 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:37.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:38.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:39.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:40.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:41.716 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:41.717 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:41.718 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:41.718 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:41.718 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:41.718 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:41.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:42.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:43.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:44.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:45.718 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:45.719 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:45.719 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:45.719 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:45.720 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:45.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:46.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:48.770 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:49.721 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:49.722 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:49.722 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:49.722 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:49.723 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:49.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:50.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:51.640 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:50:51.641 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:50:51.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:52.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:53.724 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:53.725 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:53.725 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:53.725 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:53.725 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:53.725 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:53.726 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:53.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:54.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:55.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:56.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:57.727 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:50:57.728 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:50:57.729 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:50:57.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:58.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:50:59.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:00.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:01.729 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:01.730 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:01.731 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:01.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:02.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:03.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:04.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:05.731 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:05.732 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:05.732 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:05.732 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:05.732 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.733 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:05.734 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:05.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:06.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:07.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:08.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:09.733 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:09.734 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:09.735 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:09.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:10.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:11.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:12.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:13.736 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:13.736 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:13.737 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:13.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:14.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:15.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:16.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:17.738 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:17.739 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:17.739 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:17.739 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:17.740 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:18.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:19.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:20.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:21.740 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:21.741 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:21.741 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:21.742 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:21.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:22.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:23.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:24.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:25.743 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:25.744 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:25.745 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:25.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:26.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:27.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:28.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:29.746 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:29.747 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:29.748 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:29.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:30.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:31.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:32.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:33.748 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:33.750 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:33.751 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:33.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:34.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:35.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:36.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:37.751 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:37.753 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:37.754 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:37.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:38.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:39.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:40.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:41.754 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:41.755 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:41.756 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:41.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:42.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:43.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:44.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:45.756 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:45.757 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:45.758 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:45.759 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:45.759 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:45.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:46.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:48.772 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:49.759 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:49.760 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:49.760 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:49.760 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:49.760 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:49.760 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:49.761 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:49.773 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:50.109 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:51:50.112 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:51:50.775 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:51:51.664 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:51:51.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:52.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:53.762 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:53.763 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:53.763 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:53.763 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:53.763 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:53.763 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:53.764 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:53.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:54.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:55.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:56.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:57.764 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:51:57.765 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:51:57.765 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:51:57.765 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:51:57.766 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:51:57.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:58.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:51:59.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:00.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:01.766 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:01.767 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:01.767 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:01.767 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:01.768 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:01.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:02.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:03.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:04.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:05.768 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:05.769 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:05.769 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:05.769 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:05.769 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:05.770 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:05.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:06.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:07.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:08.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:09.771 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:09.772 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:09.773 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:09.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:10.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:11.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:12.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:13.773 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:13.774 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:13.775 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:13.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:14.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:15.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:16.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:17.775 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:17.776 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:17.777 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:17.777 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:17.777 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:17.777 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:17.777 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:18.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:19.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:20.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:21.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:21.778 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:21.779 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:21.779 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:21.779 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:21.779 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:21.779 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:21.780 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:22.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:23.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:24.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:25.780 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:25.781 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:25.782 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:25.783 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:25.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:26.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:27.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:28.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:29.783 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:29.784 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:29.785 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:29.786 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:29.786 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:29.786 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:29.786 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:29.786 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:29.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:30.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:31.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:32.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:33.786 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:33.787 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:33.787 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:33.787 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:33.788 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:33.789 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:33.789 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:33.789 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:33.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:34.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:35.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:36.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:37.788 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:37.789 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:37.790 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:37.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:38.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:39.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:40.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:41.790 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:41.791 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:41.792 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:41.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:42.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:43.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:44.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:45.793 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:45.794 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:45.795 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:45.795 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:45.795 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:45.795 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:45.795 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:45.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:46.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:48.774 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:49.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:49.794 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:49.795 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:49.795 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:49.795 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:49.796 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:50.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:52:51.686 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:52:51.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:52.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:53.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:53.796 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:53.797 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:53.798 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:54.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:55.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:56.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:57.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:57.797 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:52:57.798 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:52:57.798 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:52:57.798 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:57.798 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:52:57.799 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:52:58.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:52:59.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:00.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:01.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:01.799 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:01.800 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:01.801 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:02.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:03.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:04.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:05.801 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:05.802 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:05.802 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:05.802 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:05.802 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:05.802 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:05.803 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:05.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:06.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:07.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:08.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:09.803 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:09.804 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:09.804 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:09.804 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:09.804 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:09.805 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:09.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:10.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:11.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:12.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:13.805 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:13.806 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:13.807 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:13.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:14.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:15.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:16.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:17.807 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:17.808 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:17.809 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:18.776 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:19.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:20.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:21.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:21.808 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:21.809 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:21.809 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:21.809 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:21.809 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:21.810 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:22.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:23.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:24.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:25.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:25.810 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:25.811 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:25.811 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:25.811 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:25.812 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:26.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:27.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:28.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:29.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:29.812 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:29.813 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:29.814 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:29.814 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:29.814 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:30.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:31.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:32.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:33.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:33.813 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:33.814 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:33.814 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:33.814 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:33.815 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:34.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:35.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:36.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:37.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:37.815 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:37.816 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:37.817 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:38.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:39.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:40.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:41.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:41.816 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:41.817 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:41.817 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:41.817 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:41.817 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:41.818 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:42.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:43.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:44.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:45.819 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:45.820 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:45.821 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:45.822 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:45.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:46.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:48.777 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:49.575 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:53:49.577 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:53:49.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:49.821 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:49.903 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:49.904 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:49.904 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:49.904 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:49.904 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:49.904 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:49.905 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:50.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:53:51.753 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:53:51.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:51.960 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:53:51.963 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:53:52.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:53.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:53.904 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:53.905 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:53.905 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:53.905 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:53.905 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:53.905 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:53.906 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:54.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:55.505 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:53:55.507 7fbb3ce50d60 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:53:55.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:56.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:57.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:57.906 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:53:57.907 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:53:57.907 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:53:57.908 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:53:58.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:53:59.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:00.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:01.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:01.908 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:01.909 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:01.909 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:01.909 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:01.909 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:01.909 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:01.910 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:01.911 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:02.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:03.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:04.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:05.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:05.910 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:05.911 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:05.912 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:06.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:07.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:08.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:09.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:09.912 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:09.913 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:09.914 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:09.914 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:10.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:11.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:12.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:13.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:13.913 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:13.914 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:13.915 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:13.916 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:13.916 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:14.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:15.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:16.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:17.915 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:17.916 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:17.917 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:17.917 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:17.917 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:18.778 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:19.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:20.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:21.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:21.916 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:21.917 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:21.918 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:22.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:23.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:24.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:25.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:25.918 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:25.919 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:25.920 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:26.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:27.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:28.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:29.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:29.921 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:29.922 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:29.923 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:30.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:31.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:32.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:33.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:33.922 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:33.923 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:33.924 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:33.925 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:34.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:35.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:36.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:37.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:37.924 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:37.925 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:37.926 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:38.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:39.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:40.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:41.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:41.926 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:41.927 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:41.927 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:41.927 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:41.927 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:41.928 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:42.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:43.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:44.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:45.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:45.928 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:45.929 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:45.929 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:45.929 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:45.929 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:45.930 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:46.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:48.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:49.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:49.930 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:49.931 7fbb3ce50d60 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:49.931 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:49.931 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:49.932 7fbb3ce50d60 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:50.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:54:51.778 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:54:51.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:52.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:53.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:53.932 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:53.933 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:53.933 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:53.933 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:53.934 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:54.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:55.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:56.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:57.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:57.934 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:54:57.935 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:54:57.935 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:54:57.936 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:54:58.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:54:59.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:00.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:01.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:01.937 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:01.938 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:01.938 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:01.938 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:01.939 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:01.940 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:01.940 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:02.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:03.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:04.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:05.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:05.938 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:05.940 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:05.941 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:05.941 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:05.941 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:05.941 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:05.941 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:06.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:07.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:08.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:09.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:09.940 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:09.941 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:09.941 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:09.941 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:09.942 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:10.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:11.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:12.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:13.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:13.942 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:13.943 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:13.943 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:13.943 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:13.943 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:13.943 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:13.944 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:14.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:15.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:16.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:17.944 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:17.945 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:17.946 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:18.779 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:19.781 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:20.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:21.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:21.946 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:21.947 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:21.948 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:21.948 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:21.948 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:21.948 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:21.948 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:22.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:23.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:24.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:25.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:25.947 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:25.948 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:25.948 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:25.948 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:25.949 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:26.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:27.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:28.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:29.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:29.949 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:29.950 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:29.950 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:29.951 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:29.952 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:29.952 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:30.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:31.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:32.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:33.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:33.951 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:33.952 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:33.952 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:33.953 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:34.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:35.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:36.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:37.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:37.953 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:37.954 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:37.955 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:38.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:39.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:40.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:41.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:41.955 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:41.956 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:41.956 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:41.956 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:41.956 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:41.956 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:41.957 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:42.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:43.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:44.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:45.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:45.957 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:45.958 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:45.959 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:45.960 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:45.960 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:46.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:48.780 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:49.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:49.959 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:49.960 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:49.960 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:49.960 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:49.961 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:50.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:51.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:51.800 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:55:51.800 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:55:51.801 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:55:51.801 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:55:51.801 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:55:51.801 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:55:51.801 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:55:52.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:53.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:53.961 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:53.962 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:53.963 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:54.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:55.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:56.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:57.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:57.963 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:55:57.963 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:55:57.964 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:55:58.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:55:59.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:00.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:01.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:01.964 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:01.965 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:01.966 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:02.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:03.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:04.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:05.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:05.966 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:05.967 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:05.968 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:05.968 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:05.968 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:05.968 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:06.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:07.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:08.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:09.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:09.968 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:09.969 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:09.970 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:09.970 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:09.970 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:09.970 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:09.970 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:10.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:11.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:12.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:13.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:13.969 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:13.970 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:13.970 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:13.970 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:13.970 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:13.970 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:13.971 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:14.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:15.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:16.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:17.971 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:17.972 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:17.973 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:17.973 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:18.782 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:19.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:20.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:21.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:21.973 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:21.973 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:21.974 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:21.975 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:22.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:23.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:24.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:25.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:25.975 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:25.976 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:25.977 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:25.977 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:25.977 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:25.977 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:25.977 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:26.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:27.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:28.662 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:56:28.664 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:56:28.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:29.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:29.976 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:29.977 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:29.977 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:29.977 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:29.978 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:30.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:31.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:32.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:33.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:33.978 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:33.979 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:33.980 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:33.981 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:34.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:35.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:36.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:37.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:37.981 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:37.982 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:37.983 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:38.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:39.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:40.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:41.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:41.982 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:41.983 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:41.984 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:41.985 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:41.985 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:42.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:43.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:44.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:45.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:45.984 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:45.985 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:45.985 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:45.985 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:45.986 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:46.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:48.783 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:49.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:49.986 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:49.987 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:49.987 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:49.987 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:49.987 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:49.988 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:50.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:51.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:56:51.822 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:56:52.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:53.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:53.988 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:53.988 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:53.989 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:54.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:55.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:56.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:57.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:57.989 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:56:57.990 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:56:57.990 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:56:57.991 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:56:58.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:56:59.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:00.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:01.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:01.991 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:01.992 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:01.993 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:02.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:03.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:04.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:05.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:05.993 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:05.994 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:05.994 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:05.994 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:05.995 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:06.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:07.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:08.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:09.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:09.995 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:09.996 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:09.996 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:09.996 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:09.996 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:09.997 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:10.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:11.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:12.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:13.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:13.997 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:13.998 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:13.999 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:14.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:15.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:16.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:17.999 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:18.000 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:18.001 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:18.784 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:19.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:20.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:21.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:22.000 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:22.002 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:22.003 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:22.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:23.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:24.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:25.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:26.002 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:26.003 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:26.003 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:26.003 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:26.004 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:26.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:27.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:28.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:29.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:30.004 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:30.005 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:30.006 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:30.006 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:30.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:31.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:32.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:33.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:34.005 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:34.006 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:34.006 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:34.006 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:34.007 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:34.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:35.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:36.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:37.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:38.007 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:38.008 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:38.009 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:38.010 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:38.010 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:38.010 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:38.010 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:38.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:39.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:40.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:41.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:42.009 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:42.010 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:42.010 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:42.010 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:42.010 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:42.010 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:42.011 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:42.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:43.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:44.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:45.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:46.011 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:46.012 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:46.013 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:46.013 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:46.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:48.785 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:49.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:50.012 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:50.013 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:50.013 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:50.013 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:50.014 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:50.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:51.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:51.890 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:57:51.890 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:57:51.890 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:57:51.890 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:57:51.891 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:57:51.891 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:57:51.891 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:57:52.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:53.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:54.014 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:54.015 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:54.016 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:54.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:55.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:56.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:57.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:58.015 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:57:58.016 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:57:58.016 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:57:58.016 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:57:58.017 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:57:58.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:57:59.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:00.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:01.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:02.017 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:02.018 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:02.019 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:02.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:03.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:04.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:05.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:06.018 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:06.020 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:06.021 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:06.021 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:06.021 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:06.021 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:06.021 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:06.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:07.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:08.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:09.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:10.020 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:10.021 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:10.021 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:10.021 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:10.022 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:10.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:11.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:12.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:13.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:14.022 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:14.023 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:14.024 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:14.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:15.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:16.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:18.024 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:18.025 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:18.026 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:18.786 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:19.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:20.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:21.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:22.026 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:22.027 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:22.028 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:22.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:23.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:24.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:25.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:26.027 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:26.028 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:26.029 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:26.030 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:26.030 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:26.030 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:26.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:27.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:28.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:29.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:30.030 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:30.031 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:30.032 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:30.032 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:30.032 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:30.032 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:30.032 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:30.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:31.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:32.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:33.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:34.031 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:34.032 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:34.032 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:34.032 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:34.033 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:34.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:35.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:36.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:37.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:38.033 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:38.034 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:38.034 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:38.034 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:38.034 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:38.035 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:38.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:39.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:40.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:41.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:42.035 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:42.036 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:42.037 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:42.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:43.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:44.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:45.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:46.037 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:46.038 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:46.039 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:46.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:48.787 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:49.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:50.039 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:50.040 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:50.040 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:50.040 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:50.040 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:50.040 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:50.041 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:50.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:51.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:58:51.914 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:58:52.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:53.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:54.041 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:54.042 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:54.043 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:54.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:55.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:56.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:57.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:58.043 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:58:58.044 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:58:58.045 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:58:58.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:58:59.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:00.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:01.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:02.045 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:02.046 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:02.047 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:02.048 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:02.048 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:02.048 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:02.048 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:02.048 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:02.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:03.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:04.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:05.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:06.047 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:06.048 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:06.048 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:06.048 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:06.049 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:06.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:07.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:08.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:09.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:10.049 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:10.050 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:10.050 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:10.051 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:10.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:11.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:12.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:13.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:14.051 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:14.052 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:14.052 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:14.052 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:14.053 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:14.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:15.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:16.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:18.053 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:18.054 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:18.055 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:18.788 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:19.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:20.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:21.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:22.055 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:22.056 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.057 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:22.057 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.057 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.057 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:22.057 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:22.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:23.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:24.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:25.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:26.057 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:26.058 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:26.059 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:26.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:27.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:28.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:29.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:30.059 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:30.060 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:30.060 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:30.060 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:30.061 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:30.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:30.892 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:59:30.895 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:59:31.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:32.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:33.217 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:59:33.219 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:59:33.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:34.061 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:34.062 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:34.063 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:34.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:35.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:36.791 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 01:59:36.794 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 01:59:36.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:37.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:38.063 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:38.064 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:38.065 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.066 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:38.066 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.066 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.066 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:38.066 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:38.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:39.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:40.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:41.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:42.066 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.067 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.068 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:42.068 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:42.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:43.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:44.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:45.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:46.067 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:46.069 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:46.070 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:46.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:48.789 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:49.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:50.069 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:50.070 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:50.070 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:50.070 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:50.071 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:50.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:51.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 01:59:51.937 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 01:59:52.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:53.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:54.071 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:54.072 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:54.073 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:54.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:55.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:56.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:57.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:58.072 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 01:59:58.073 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 01:59:58.073 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 01:59:58.073 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 01:59:58.074 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 01:59:58.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 01:59:59.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:00.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:01.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:02.074 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:02.076 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:02.077 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:02.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:03.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:04.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:05.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:06.076 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:06.077 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:06.078 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:06.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:07.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:08.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:09.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:10.078 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:10.079 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:10.080 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:10.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:11.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:12.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:13.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:14.079 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:14.080 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.081 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.082 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:14.082 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:14.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:15.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:16.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:18.082 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:18.083 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:18.084 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:18.790 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:19.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:20.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:21.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:22.083 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:22.084 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:22.084 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:22.084 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:22.085 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:22.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:23.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:24.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:25.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:26.086 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:26.087 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.088 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:26.088 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.088 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.088 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:26.088 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:26.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:27.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:28.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:29.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:30.088 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:30.089 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.090 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.090 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:30.090 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:30.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:31.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:32.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:33.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:34.089 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:34.090 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:34.090 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:34.090 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:34.090 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:34.091 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:34.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:35.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:36.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:37.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:38.091 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:38.092 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.093 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:38.093 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.093 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.093 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:38.093 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:38.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:39.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:40.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:41.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:42.092 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:42.093 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:42.093 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:42.093 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:42.094 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:42.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:43.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:44.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:45.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:46.094 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:46.095 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:46.096 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.097 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.097 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:46.097 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:46.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:48.791 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:49.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:50.096 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:50.097 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:50.097 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:50.097 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:50.098 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:50.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:51.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:51.959 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:00:51.959 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:00:51.959 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:00:51.959 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:00:51.960 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:00:51.960 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:00:51.960 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:00:52.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:53.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:54.098 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:54.100 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:54.101 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:54.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:55.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:56.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:57.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:58.100 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:00:58.102 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.103 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:00:58.103 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.103 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.103 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:00:58.103 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:00:58.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:00:59.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:00.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:01.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:02.103 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:02.104 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:02.104 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:02.104 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:02.104 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:02.104 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:02.105 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:02.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:03.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:04.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:05.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:06.105 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:06.106 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:06.107 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:06.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:07.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:08.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:09.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:10.107 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:10.108 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:10.108 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:10.108 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.109 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:10.110 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:10.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:11.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:12.859 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:13.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:14.109 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:14.111 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:14.112 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:14.861 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:15.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:16.864 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:18.112 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:18.113 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:18.113 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.114 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.115 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:18.115 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:18.792 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:19.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:20.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:21.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:22.115 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:22.116 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:22.117 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:22.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:23.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:24.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:25.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:26.116 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:26.117 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:26.118 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:26.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:27.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:28.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:29.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:30.118 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:30.119 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:30.119 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:30.119 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:30.120 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:30.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:31.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:32.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:33.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:34.120 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:34.121 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:34.122 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:34.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:35.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:36.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:37.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:38.122 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:38.123 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.124 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:38.125 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.125 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.125 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:38.125 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:38.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:39.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:40.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:41.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:42.124 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:42.125 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.126 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.127 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:42.127 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:42.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:43.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:44.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:45.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:46.127 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:46.128 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:46.129 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:46.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:48.793 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:49.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:50.128 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:50.129 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:50.129 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:50.130 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:50.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:51.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:01:52.042 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:01:52.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:53.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:54.130 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:54.131 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:54.132 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:54.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:55.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:56.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:57.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:58.132 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:01:58.133 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:01:58.133 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:01:58.133 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:01:58.134 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:01:58.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:01:59.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:00.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:01.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:02.134 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:02.135 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:02.135 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.136 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:02.137 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.137 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.137 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:02.137 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:02.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:02.853 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:02:02.855 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:02:03.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:04.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:05.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:06.136 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:06.137 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.138 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.138 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:06.138 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:06.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:07.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:08.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:09.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:10.138 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:10.164 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:10.164 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:10.164 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:10.164 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:10.165 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:10.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:11.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:12.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:13.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:14.165 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:14.166 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:14.167 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:14.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:15.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:16.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:18.167 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:18.168 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:18.169 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:18.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:19.795 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:20.797 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:21.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:22.169 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:22.170 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:22.170 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:22.170 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:22.171 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:22.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:23.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:24.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:25.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:26.171 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:26.172 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:26.172 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:26.172 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.173 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.174 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:26.174 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:26.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:27.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:28.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:29.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:30.173 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:30.174 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:30.175 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:30.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:31.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:32.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:33.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:34.175 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:34.176 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.177 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.177 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:34.177 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:34.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:35.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:36.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:37.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:38.177 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:38.178 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:38.179 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:38.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:39.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:40.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:41.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:42.178 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:42.179 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.180 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:42.181 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:42.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:43.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:44.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:45.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:46.180 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:46.181 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:46.182 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:46.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:48.794 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:49.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:50.182 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:50.183 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:50.184 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:50.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:51.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:02:52.065 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:02:52.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:53.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:54.183 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:54.184 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:54.185 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:54.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:55.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:56.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:57.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:58.185 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:02:58.186 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:02:58.187 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:02:58.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:02:59.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:00.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:01.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:02.187 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:02.188 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:02.189 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:02.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:03.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:04.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:05.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:06.188 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:06.189 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:06.189 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:06.189 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:06.190 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:06.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:07.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:08.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:09.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:10.190 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:10.191 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:10.191 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:10.191 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:10.192 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:10.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:11.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:12.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:13.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:14.192 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:14.193 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:14.193 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:14.194 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:14.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:15.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:16.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:18.194 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:18.195 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:18.196 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:18.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:19.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:20.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:21.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:22.195 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:22.196 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:22.196 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:22.196 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:22.197 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:22.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:23.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:24.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:25.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:26.198 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:26.199 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.200 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:26.200 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.200 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.200 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:26.200 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:26.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:27.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:28.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:29.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:30.199 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:30.200 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:30.200 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:30.201 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:30.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:31.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:32.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:33.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:34.201 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:34.202 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:34.202 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:34.203 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:34.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:35.695 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:03:35.698 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:35.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:36.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:37.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:38.112 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:03:38.115 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:38.203 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:38.204 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:38.205 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:38.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:39.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:40.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:41.701 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:03:41.704 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:03:41.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:42.206 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:42.207 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:42.207 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:42.207 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:42.207 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:42.208 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:42.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:43.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:44.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:45.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:46.208 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:46.209 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:46.209 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:46.209 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:46.209 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:46.209 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:46.210 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:46.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:48.796 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:49.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:50.210 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:50.211 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.212 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:50.212 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.212 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.212 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:50.212 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:50.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:51.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:03:52.088 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:03:52.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:53.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:54.212 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:54.213 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:54.214 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.215 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:54.215 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.215 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.215 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:54.215 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:54.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:55.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:56.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:57.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:58.214 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:03:58.215 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:03:58.215 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:03:58.215 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:58.215 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:03:58.217 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:03:58.217 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:03:58.217 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:03:58.218 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:03:58.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:03:59.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:00.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:01.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:02.216 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:02.217 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:02.217 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:02.217 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:02.218 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:02.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:03.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:04.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:05.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:06.218 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:06.219 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:06.219 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:06.219 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:06.220 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:06.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:07.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:08.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:09.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:10.220 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.221 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:10.222 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.222 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.222 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:10.222 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:10.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:11.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:12.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:13.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:14.221 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:14.222 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:14.223 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:14.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:15.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:16.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:18.223 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:18.224 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:18.225 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:18.798 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:19.800 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:20.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:21.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:22.225 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:22.226 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:22.227 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:22.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:23.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:24.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:25.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:26.227 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:26.228 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:26.228 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:26.229 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:26.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:27.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:28.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:29.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:30.228 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:30.229 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:30.229 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:30.230 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:30.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:31.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:32.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:33.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:34.230 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:34.231 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:34.232 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:34.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:35.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:36.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:37.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:38.232 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:38.233 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:38.233 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:38.233 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:38.233 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:38.233 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:38.234 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:38.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:39.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:40.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:41.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:42.234 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:42.235 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:42.235 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:42.236 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:42.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:43.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:44.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:45.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:46.236 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:46.237 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:46.238 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:46.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:48.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:49.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:50.238 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:50.239 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:50.240 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:50.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:51.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:04:52.112 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:04:52.113 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:04:52.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:53.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:54.241 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:54.242 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:54.243 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:54.243 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:54.243 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:54.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:55.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:56.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:57.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:58.242 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:04:58.243 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:04:58.244 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:04:58.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:04:59.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:00.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:01.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:02.244 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:02.245 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:02.246 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:02.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:03.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:04.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:05.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:06.246 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:06.247 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:06.247 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:06.247 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:06.248 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:06.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:07.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:08.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:09.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:10.248 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:10.249 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:10.250 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:10.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:11.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:12.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:13.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:14.249 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:14.250 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:14.251 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:14.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:15.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:16.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:18.252 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:18.253 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:18.253 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:18.253 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:18.254 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:18.255 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:18.799 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:19.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:20.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:21.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:22.254 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:22.255 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:22.256 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:22.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:23.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:24.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:25.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:26.256 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:26.257 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:26.257 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:26.257 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:26.257 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:26.258 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:26.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:27.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:28.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:29.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:30.258 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:30.259 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:30.260 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:30.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:31.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:32.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:33.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:34.260 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:34.261 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:34.262 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:34.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:35.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:36.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:37.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:38.262 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:38.263 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:38.263 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:38.263 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:38.264 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:38.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:39.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:40.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:41.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:42.265 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:42.266 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:42.267 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:42.267 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:42.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:43.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:44.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:45.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:46.266 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:46.267 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:46.267 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:46.267 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:46.268 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:46.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:48.801 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:49.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:50.268 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:50.269 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:50.269 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:50.270 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:50.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:51.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:05:52.135 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:05:52.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:53.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:54.271 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:54.272 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:54.272 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:54.272 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:54.272 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:54.273 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:54.274 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:54.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:55.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:56.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:57.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:58.273 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:05:58.274 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:05:58.275 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:05:58.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:05:59.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:00.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:01.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:02.275 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:02.276 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:02.276 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:02.276 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:02.277 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:02.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:03.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:04.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:05.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:06.277 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:06.278 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:06.279 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:06.280 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:06.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:07.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:08.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:09.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:10.279 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:10.280 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:10.281 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:10.282 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:10.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:11.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:12.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:13.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:14.281 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:14.282 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:14.283 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:14.284 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:14.284 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:14.284 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:14.284 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:14.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:15.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:16.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:18.284 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:18.285 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:18.285 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:18.285 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:18.285 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:18.285 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:18.286 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:18.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:19.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:20.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:21.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:22.286 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:22.286 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:22.287 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:22.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:23.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:24.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:25.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:26.288 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:26.289 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:26.289 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:26.289 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:26.289 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:26.289 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:26.290 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:26.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:27.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:28.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:29.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:30.290 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:30.291 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:30.292 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:30.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:31.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:32.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:33.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:34.292 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:34.293 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:34.294 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:34.294 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:34.294 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:34.294 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:34.294 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:34.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:35.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:36.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:37.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:37.892 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:06:37.895 7fbb3cfbb360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:06:38.294 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:38.295 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:38.296 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:38.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:39.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:40.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:41.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:42.296 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:42.297 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:42.297 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:42.297 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:42.297 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:42.297 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:42.298 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:42.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:43.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:44.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:45.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:46.298 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:46.300 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.301 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:46.301 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.301 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.301 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:46.301 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:46.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:48.802 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:49.804 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:50.301 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:50.302 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:50.303 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:50.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:51.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:06:52.200 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:06:52.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:53.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:54.303 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:54.304 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:54.305 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:54.306 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:54.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:55.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:56.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:57.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:58.305 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:06:58.306 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:06:58.307 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:06:58.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:06:59.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:00.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:01.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:02.307 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:02.308 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:02.308 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:02.308 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:02.308 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:02.309 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:02.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:03.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:04.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:05.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:06.309 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:06.310 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:06.311 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:06.312 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:06.312 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:06.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:07.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:08.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:09.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:10.312 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:10.313 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:10.314 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:10.314 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:10.314 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:10.314 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:10.314 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:10.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:11.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:12.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:13.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:14.314 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:14.315 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:14.315 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:14.315 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:14.316 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:14.317 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:14.317 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:14.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:15.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:16.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:18.316 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:18.317 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:18.317 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:18.317 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:18.317 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:18.318 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:18.803 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:19.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:20.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:21.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:22.318 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:22.318 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:22.319 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:22.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:23.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:24.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:25.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:26.319 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:26.320 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:26.320 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:26.320 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:26.320 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:26.321 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:26.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:27.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:28.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:29.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:30.321 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:30.323 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:30.324 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:30.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:31.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:32.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:33.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:34.324 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:34.325 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:34.325 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:34.325 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:34.325 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:34.325 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:34.326 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:34.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:35.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:36.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:37.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:38.326 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:38.326 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:38.327 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:38.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:39.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:40.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:41.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:42.327 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:42.328 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:42.329 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:42.330 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:42.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:43.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:44.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:45.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:46.329 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:46.330 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:46.331 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:46.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:48.805 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:49.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:50.331 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:50.332 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:50.333 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:50.334 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:50.334 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:50.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:51.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:52.222 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:07:52.223 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:07:52.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:53.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:54.334 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:54.335 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:54.336 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:54.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:55.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:56.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:57.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:58.335 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:07:58.336 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:07:58.337 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:07:58.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:07:59.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:00.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:01.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:02.337 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:02.338 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:02.338 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:02.338 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:02.338 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:02.339 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:02.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:03.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:04.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:05.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:06.339 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:06.340 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:06.341 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:06.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:07.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:08.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:09.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:10.341 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:10.342 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:10.342 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:10.342 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:10.343 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:10.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:11.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:12.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:13.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:14.343 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:14.345 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.346 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:14.346 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.346 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.346 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:14.346 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:14.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:15.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:16.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:18.346 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:18.347 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:18.347 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:18.347 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:18.347 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:18.348 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:18.806 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:19.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:20.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:21.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:22.348 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:22.349 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:22.350 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:22.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:23.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:24.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:25.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:26.350 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:26.351 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:26.351 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:26.351 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:26.352 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:26.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:27.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:28.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:29.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:30.352 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:30.353 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:30.353 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:30.353 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:30.353 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:30.354 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:30.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:31.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:32.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:33.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:34.354 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:34.355 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:34.355 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:34.355 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:34.356 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:34.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:35.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:36.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:37.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:38.357 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:38.358 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:38.358 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:38.358 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:38.358 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:38.359 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:38.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:39.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:40.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:41.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:42.359 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:42.360 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:42.360 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:42.360 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:42.361 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:42.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:43.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:44.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:45.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:46.361 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:46.362 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:46.362 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:46.362 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:46.363 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:46.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:48.807 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:49.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:50.364 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:50.365 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:50.365 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:50.365 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:50.365 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:50.366 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:50.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:51.811 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:52.244 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:08:52.244 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:08:52.244 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:08:52.245 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:08:52.245 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:08:52.245 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:08:52.245 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:08:52.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:53.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:54.366 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:54.367 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:54.368 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:54.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:55.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:56.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:57.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:58.368 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:08:58.370 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:58.371 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:08:58.371 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:58.371 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:58.371 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:08:58.371 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:08:58.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:08:59.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:00.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:01.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:02.371 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:02.372 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:02.373 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:02.374 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:02.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:03.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:04.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:05.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:06.373 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:06.374 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:06.375 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:06.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:07.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:08.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:09.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:10.374 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:10.375 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:10.375 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:10.375 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:10.376 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:10.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:11.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:12.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:13.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:14.376 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:14.377 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:14.378 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:14.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:15.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:16.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:18.378 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:18.379 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:18.380 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:18.380 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:18.381 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:18.808 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:19.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:20.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:20.963 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:09:20.966 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:21.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:22.380 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:22.381 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:22.381 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:22.381 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:22.382 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:22.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:23.389 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:09:23.392 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:23.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:24.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:25.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:26.382 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:26.383 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:26.383 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:26.383 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:26.384 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:26.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:26.923 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:09:26.925 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:09:27.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:28.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:29.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:30.384 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:30.385 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:30.385 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:30.385 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:30.385 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:30.386 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:30.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:31.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:32.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:33.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:34.386 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:34.387 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:34.387 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:34.387 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:34.388 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:34.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:35.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:36.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:37.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:38.388 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:38.389 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:38.390 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:38.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:39.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:40.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:41.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:42.390 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:42.392 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:42.393 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:42.393 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:42.393 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:42.393 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:42.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:43.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:44.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:45.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:46.392 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:46.393 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:46.394 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.395 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:46.395 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.395 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.395 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:46.395 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:46.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:48.809 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:49.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:50.394 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:50.396 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:50.397 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:50.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:51.814 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:09:52.266 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:09:52.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:53.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:54.397 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:54.398 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:54.398 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:54.398 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:54.398 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:54.399 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:54.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:55.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:56.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:57.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:58.399 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:09:58.400 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:09:58.401 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:09:58.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:09:59.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:00.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:01.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:02.401 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:02.402 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:02.402 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:02.402 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:02.402 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:02.403 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:02.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:03.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:04.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:05.857 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:06.402 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:06.403 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:06.404 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:06.405 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:06.405 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:06.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:07.860 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:08.862 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:09.864 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:10.404 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:10.405 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:10.405 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:10.405 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:10.405 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:10.406 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:10.865 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:11.867 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:12.868 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:13.870 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:14.406 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:14.407 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:14.407 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:14.407 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:14.408 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:14.872 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:15.873 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:16.875 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:18.408 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:18.409 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:18.409 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:18.409 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:18.410 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:18.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:19.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:20.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:21.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:22.410 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:22.411 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:22.411 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:22.411 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:22.411 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:22.411 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:22.412 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:22.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:23.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:24.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:25.822 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:26.413 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:26.414 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:26.414 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:26.414 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.415 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:26.416 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:26.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:27.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:28.827 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:29.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:30.416 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:30.417 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:30.418 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:30.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:31.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:32.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:33.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:34.418 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:34.419 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:34.420 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:34.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:35.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:36.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:37.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:38.420 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:38.421 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:38.422 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:38.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:39.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:40.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:41.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:42.422 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:42.424 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.425 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:42.425 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.425 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.425 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:42.425 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:42.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:43.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:44.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:45.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:46.424 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:46.425 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:46.425 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:46.425 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:46.426 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:46.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:48.810 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:49.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:50.427 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:50.428 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.429 7fbb3cfbb360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.429 7fbb3cfbb360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:50.429 7fbb3cfbb360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:50.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:51.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:10:52.287 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:10:52.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:53.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:54.429 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:54.430 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:54.430 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:54.430 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:54.431 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:54.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:55.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:56.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:57.824 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:58.431 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:10:58.432 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.433 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.433 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:10:58.433 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:10:58.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:10:59.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:00.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:01.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:02.433 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:02.434 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:02.435 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:02.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:03.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:04.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:05.835 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:06.434 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:06.435 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.436 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:06.437 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:06.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:07.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:08.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:09.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:10.436 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:10.438 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:10.439 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:10.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:11.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:12.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:13.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:14.439 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:14.440 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:14.441 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:14.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:15.852 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:16.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:18.441 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:18.442 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:18.442 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:18.442 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:18.443 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:18.812 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:19.861 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:20.863 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:21.864 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:22.443 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.445 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:22.446 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.446 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.446 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:22.446 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:22.865 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:23.866 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:24.867 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:25.871 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:26.446 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.447 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.448 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:26.448 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:26.911 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:27.200 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [none:ConfigApiWorkerTrace] map 09/01/2025 02:11:27.203 7fbb3df45360 [contrail-api] [DEBUG]: send sync_uve in the [ObjectConfigNode:NodeStatusUVE] map 09/01/2025 02:11:27.913 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:28.914 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:29.915 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:30.448 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:30.449 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:30.450 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:30.916 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:31.918 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:32.919 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:33.920 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:34.450 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:34.451 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:34.452 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:34.922 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:35.923 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:36.924 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:37.926 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:38.452 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:38.453 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:38.454 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.455 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:38.455 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.455 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.455 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:38.455 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:38.928 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:39.929 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:40.930 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:41.931 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:42.455 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:42.456 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:42.457 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:42.933 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:43.934 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:44.936 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:45.938 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:46.457 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:46.458 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:46.458 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:46.459 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:46.939 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:48.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:49.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:50.460 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:50.461 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.462 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:50.462 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.462 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:50.462 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:50.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:51.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field virtual_machine_interfaces in resource virtual-router: 'virtual_machine_interfaces' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field network_ipam_refs in resource virtual-router: 'network_ipam_refs' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field sub_cluster_refs in resource virtual-router: 'sub_cluster_refs' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field tag_refs in resource virtual-router: 'tag_refs' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field instance_ip_back_refs in resource virtual-router: 'instance_ip_back_refs' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field physical_router_back_refs in resource virtual-router: 'physical_router_back_refs' 09/01/2025 02:11:52.354 7fbb3d0e3900 [contrail-api] [WARNING]: __default__ [SYS_WARN]: VncApiDebug: Key not found for field provider_attachment_back_refs in resource virtual-router: 'provider_attachment_back_refs' 09/01/2025 02:11:52.819 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:53.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:54.462 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:54.463 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:54.463 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:54.463 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:54.463 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:54.463 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:54.464 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:54.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:55.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:56.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:57.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:58.464 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:11:58.465 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:11:58.466 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:11:58.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:11:59.829 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:00.831 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:01.833 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:02.467 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:02.468 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.469 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.469 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:02.469 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:02.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:03.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:04.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:05.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:06.469 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:06.470 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:06.471 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:06.840 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:07.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:08.842 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:09.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:10.470 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:10.471 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:10.471 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:10.472 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:10.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:11.846 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:12.848 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:13.883 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:14.473 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:14.475 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:14.476 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:14.885 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:15.886 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:16.887 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:18.476 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:18.477 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:18.477 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:18.477 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.478 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:18.479 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:18.813 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:19.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:20.817 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:21.818 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:22.478 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:22.479 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:22.479 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:22.479 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:22.479 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:22.480 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:22.820 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:23.821 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:24.823 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:25.825 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:26.481 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:26.482 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:26.482 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:26.482 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.483 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.484 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:26.484 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:26.826 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:27.828 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:28.830 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:29.832 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:30.483 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:30.484 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:30.485 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:30.834 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:31.836 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:32.837 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:33.838 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:34.485 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:34.486 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.487 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:34.487 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.487 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.487 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:34.487 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:34.839 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:35.841 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:36.843 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:37.844 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:38.487 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:38.488 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:38.489 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:38.845 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:39.847 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:40.849 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:41.850 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:42.489 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:42.490 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:42.490 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:42.490 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:42.491 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:42.851 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:43.853 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:44.854 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:45.856 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:46.491 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:46.492 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:46.492 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:46.492 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Connect] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvTcpConnectFail] in state[Connect] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvTcpConnectFail] => State[Connect] -> State[Idle] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1 process_status = [ << module_id = contrail-api instance_id = 0 state = Non-Functional connection_infos = [ << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Down description = Connect to Idle on EvTcpConnectFail >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Up description = >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[Connect to Idle on EvTcpConnectFail] connection down >>, ] >> 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-juju-k8s-1454-1:Config:contrail-api:0 client_info = << status = Idle successful_connections = 0 pid = 1 http_port = 8084 start_time = 1756690481130331 collector_name = collector_ip = 10.0.0.50:8086 collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 4 >> 09/01/2025 02:12:46.493 7fbb3df45360 [contrail-api] [INFO]: Discarding event[EvSandeshUVESend] in state[Idle] 09/01/2025 02:12:46.858 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:48.815 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:49.816 7fbb3d0e3400 [contrail-api] [DEBUG]: __default__ [SYS_DEBUG]: VncApiError: KombuAmqpClient: socket.timeout occurred while draining events, no events to process. 09/01/2025 02:12:50.493 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvIdleHoldTimerExpired] in state[Idle] 09/01/2025 02:12:50.494 7fbb3df45360 [contrail-api] [ERROR]: Session Event: TCP Connect Fail 09/01/2025 02:12:50.494 7fbb3df45360 [contrail-api] [INFO]: Sandesh Client: Event[EvIdleHoldTimerExpired] => State[Idle] -> State[Connect] 09/01/2025 02:12:50.494 7fbb3df45360 [contrail-api] [INFO]: Processing event[EvSandeshUVESend] in state[Connect]