02/03/2025 01:55:10.608 7fbee2cbb630 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:10.609 7fbee2cbb630 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local:Config:contrail-schema:0 client_info = << status = Idle successful_connections = 0 pid = 1 start_time = 1738547710608698 collector_name = collector_ip = collector_list = [ 10.0.0.40:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 02/03/2025 01:55:10.719 7fbee3e75e88 [contrail-schema] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.40:5673//'] 02/03/2025 01:55:10.742 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:10.743 7fbee19260f0 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:10.743 7fbee19260f0 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:10.765 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:10.765 7fbee19260f0 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:10.809 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ publish connection down 02/03/2025 01:55:10.812 7fbee1402240 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:10.828 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 01:55:10.828 7fbeea686978 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:10.849 7fbee141d048 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[] connection down >>, ] >> 02/03/2025 01:55:10.875 7fbee3e75e88 [contrail-schema] [WARNING]: Waiting to be elected as master... 02/03/2025 01:55:10.877 7fbee18526d8 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:10.903 7fbee3e75e88 [contrail-schema] [WARNING]: Elected master Schema Transformer node. Initializing... 02/03/2025 01:55:10.952 7fbee1867a20 [contrail-schema] [ERROR]: Cannot write http_port 8087 to /tmp/contrail-schema.0.http_port 02/03/2025 01:55:10.952 7fbee1867a20 [contrail-schema] [ERROR]: Starting Introspect on HTTP Port 8087 02/03/2025 01:55:10.953 7fbee1867c18 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Initializing description = ApiServer >>, ] description = Collector:Collector[none to Idle on EvStart], ApiServer:ApiServer[ApiServer] connection down >>, ] >> 02/03/2025 01:55:10.963 7fbee3e75e88 [contrail-schema] [WARNING]: VNCCassandra started with driver cql 02/03/2025 01:55:10.963 7fbee3e75e88 [contrail-schema] [WARNING]: initializing cassandra cluster 02/03/2025 01:55:10.963 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster before create 02/03/2025 01:55:10.964 7fbee141db70 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:10.964 7fbee141db70 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/03/2025 01:55:10.965 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster created 02/03/2025 01:55:11.014 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 02/03/2025 01:55:11.017 7fbee04a6588 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Initializing description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Zookeeper:Zookeeper[], Database:Cassandra[] connection down >>, ] >> 02/03/2025 01:55:11.024 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 02/03/2025 01:55:11.024 7fbee04f9438 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 02/03/2025 01:55:11.035 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.118 7fbee3e75e88 [contrail-schema] [WARNING]: keyspace 'to_bgp_keyspace', created 02/03/2025 01:55:11.123 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.130 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.199 7fbee3e75e88 [contrail-schema] [WARNING]: keyspace 'to_bgp_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 02/03/2025 01:55:11.201 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.201 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 02/03/2025 01:55:11.318 7fbee3e75e88 [contrail-schema] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 02/03/2025 01:55:11.328 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.536 7fbee3e75e88 [contrail-schema] [WARNING]: table 'route_target_table', created 02/03/2025 01:55:11.543 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.556 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.668 7fbee3e75e88 [contrail-schema] [WARNING]: table 'route_target_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 01:55:11.671 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.677 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.811 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_ip_address_table', created 02/03/2025 01:55:11.815 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.819 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:11.880 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_ip_address_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 01:55:11.889 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:11.898 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:12.109 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_table', created 02/03/2025 01:55:12.117 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:12.130 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:12.238 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 01:55:12.243 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:12.253 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:12.396 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_uuid_table', created 02/03/2025 01:55:12.400 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:12.406 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock acquired. Performing schema change... 02/03/2025 01:55:12.474 7fbee3e75e88 [contrail-schema] [WARNING]: table 'service_chain_uuid_table' fixed with properties {'gc_grace_seconds': 864000} 02/03/2025 01:55:12.480 7fbee3e75e88 [contrail-schema] [WARNING]: Zookeeper lock released. 02/03/2025 01:55:12.481 7fbee04a66d8 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:12.505 7fbee3e75e88 [contrail-schema] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.40:5673//'] 02/03/2025 01:55:12.506 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:12.506 7fbee04472e8 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:12.506 7fbee04472e8 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Down description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:12.511 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:12.511 7fbee044f390 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:12.517 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ publish connection down 02/03/2025 01:55:12.517 7fbee044e4e0 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Down description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:12.521 7fbee3e75e88 [contrail-schema] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 01:55:12.522 7fbee044e438 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:12.523 7fbee044fac8 [contrail-schema] [WARNING]: RabbitMQ publish connection down 02/03/2025 01:55:12.524 7fbee0448828 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Down description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 02/03/2025 01:55:12.538 7fbee044fac8 [contrail-schema] [WARNING]: RabbitMQ publish connection ESTABLISHED 02/03/2025 01:55:12.539 7fbee044e630 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 02/03/2025 01:55:14.613 7fbee044ee10 [contrail-schema] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-132-1.cluster.local process_status = [ << module_id = contrail-schema instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.40:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.40:5673, ] status = Up description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.40:2181, ] status = Up description = >>, << type = ApiServer name = ApiServer server_addrs = [ 10.0.0.40,:8082, ] status = Up description = ApiServer >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.40:9041, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired] connection down >>, ] >> 02/03/2025 01:55:15.667 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:default-project:__link_local__): Duration (0.001) sec 02/03/2025 01:55:15.668 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:default-project:dci-network): Duration (0.000) sec 02/03/2025 01:55:15.668 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:default-project:default-virtual-network): Duration (0.000) sec 02/03/2025 01:55:15.669 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:default-project:ip-fabric), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.001) sec 02/03/2025 01:55:15.810 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:default-project:ip-fabric): Duration (0.141) sec 02/03/2025 01:55:15.810 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:15.814 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:16.168 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.358) sec 02/03/2025 01:55:16.169 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:16.169 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:16.169 7fbee3e75e88 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:16.457 7fbee3e75e88 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.288) sec 02/03/2025 01:55:17.291 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.291 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.323 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:17.337 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.337 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.355 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:17.386 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.386 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.399 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:17.413 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.413 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.433 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:17.462 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.462 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.483 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:17.504 7fbee044ef60 [contrail-schema] [ERROR]: Error in rabbitmq drainer greenlet: 'SchemaTransformerLogger' object has no attribute 'warn' 02/03/2025 01:55:17.504 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection down 02/03/2025 01:55:17.586 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:default-project:ip-fabric), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:17.654 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:default-project:ip-fabric): Duration (0.069) sec 02/03/2025 01:55:17.654 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:17.655 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:17.655 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:17.732 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.078) sec 02/03/2025 01:55:17.732 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:17.732 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:17.733 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:17.899 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.167) sec 02/03/2025 01:55:18.133 7fbee044ef60 [contrail-schema] [WARNING]: RabbitMQ drainer connection ESTABLISHED 02/03/2025 01:55:19.834 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:19.836 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:19.836 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:19.837 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.003) sec 02/03/2025 01:55:19.899 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.002) sec 02/03/2025 01:55:19.899 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:19.899 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:19.916 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.019) sec 02/03/2025 01:55:20.080 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:20.080 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:20.080 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:20.081 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.001) sec 02/03/2025 01:55:25.101 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:25.110 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:25.110 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:25.115 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.015) sec 02/03/2025 01:55:25.252 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:25.253 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:25.253 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:25.253 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-service-network): Duration (0.001) sec 02/03/2025 01:55:33.490 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:33.490 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:33.490 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:33.491 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.004) sec 02/03/2025 01:55:33.537 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:33.537 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:33.537 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:33.538 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.002) sec 02/03/2025 01:55:33.814 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:33.814 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:33.814 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:33.815 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.002) sec 02/03/2025 01:55:34.815 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:34.815 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:34.815 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:34.816 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.001) sec 02/03/2025 01:55:34.833 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:34.834 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.001) sec 02/03/2025 01:55:34.835 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:34.836 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.002) sec 02/03/2025 01:55:35.038 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-service-np): Duration (0.000) sec 02/03/2025 01:55:35.039 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-ip-fabric-np): Duration (0.000) sec 02/03/2025 01:55:35.039 7fbee044ef60 [contrail-schema] [WARNING]: Network Policy Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network), Policy Name (default-domain:k8s-default:k8s-default-pod-service-np): Duration (0.000) sec 02/03/2025 01:55:35.049 7fbee044ef60 [contrail-schema] [WARNING]: Virtual-Network Evaluate Complete: VN Name (default-domain:k8s-default:k8s-default-pod-network): Duration (0.010) sec