07/20/2026 01:41:41.832 7f0301c315e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/20/2026 01:41:41.833 7f0301c315e0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] SandeshModuleClientTrace: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local:Config:contrail-device-manager:0 client_info = << status = Idle successful_connections = 0 pid = 110 start_time = 1784511701832761 collector_name = collector_ip = collector_list = [ 10.0.0.50:8086, ] >> sm_queue_count = 1 max_sm_queue_count = 1 >> 07/20/2026 01:41:42.174 7f03065db5c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//'] 07/20/2026 01:41:42.183 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/20/2026 01:41:42.184 7f0301c31c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/20/2026 01:41:42.184 7f0301c31c20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/20/2026 01:41:42.203 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/20/2026 01:41:42.204 7f0301c31ea0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/20/2026 01:41:42.255 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/20/2026 01:41:42.256 7f0301fa07c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Down description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:RabbitMQ[] connection down >>, ] >> 07/20/2026 01:41:42.269 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/20/2026 01:41:42.273 7f0301fa07c0 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart] connection down >>, ] >> 07/20/2026 01:41:42.290 7f03065db5c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/20/2026 01:41:42.290 7f03065db5c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/20/2026 01:41:42.291 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/20/2026 01:41:42.297 7f0301c7a040 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/20/2026 01:41:42.312 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/20/2026 01:41:42.741 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/20/2026 01:41:42.754 7f0301c7ac20 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Initializing description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[], Zookeeper:Zookeeper[] connection down >>, ] >> 07/20/2026 01:41:42.784 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/20/2026 01:41:42.785 7f0301c7ab80 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ , ] status = Down description = none to Idle on EvStart >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[none to Idle on EvStart], Database:Cassandra[] connection down >>, ] >> 07/20/2026 01:41:43.865 7f03065db5c0 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/20/2026 01:41:44.943 7f03065db5c0 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/20/2026 01:41:45.834 7f0301cc0360 [contrail-device-manager] [ERROR]: SANDESH: [DROP: WrongClientSMState] NodeStatusUVE: data = << name = cn-jenkins-deploy-platform-k8s-manifests-423-1.cluster.local process_status = [ << module_id = contrail-device-manager instance_id = 0 state = Non-Functional connection_infos = [ << type = Collector name = Collector server_addrs = [ 10.0.0.50:8086, ] status = Initializing description = Idle to Connect on EvIdleHoldTimerExpired >>, << type = Database name = RabbitMQ server_addrs = [ 10.0.0.50:5673, ] status = Up description = >>, << type = Database name = Cassandra server_addrs = [ 10.0.0.50:9041, ] status = Initializing description = >>, << type = Zookeeper name = Zookeeper server_addrs = [ 10.0.0.50:2181, ] status = Up description = >>, ] description = Collector:Collector[Idle to Connect on EvIdleHoldTimerExpired], Database:Cassandra[] connection down >>, ] >> 07/20/2026 01:41:45.970 7f03065db5c0 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/20/2026 01:41:47.114 7f03065db5c0 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/20/2026 01:41:48.208 7f03065db5c0 [contrail-device-manager] [WARNING]: Waiting for keyspace config_db_uuid to be created 07/20/2026 01:41:48.223 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/20/2026 01:41:48.308 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/20/2026 01:41:48.909 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:49.188 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table', created 07/20/2026 01:41:49.211 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:49.891 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:49.993 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'obj_fq_name_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:50.001 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:50.833 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:50.837 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'obj_shared_table', created 07/20/2026 01:41:50.844 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:51.560 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:51.702 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'obj_shared_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:51.705 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:52.233 7f03065db5c0 [contrail-device-manager] [ERROR]: ZTP manager: Error restarting dnsmasq DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))") 07/20/2026 01:41:52.735 7f03065db5c0 [contrail-device-manager] [WARNING]: Process 110 prepared to run in Full mode... 07/20/2026 01:41:52.735 7f03065db5c0 [contrail-device-manager] [WARNING]: Elected master Device Manager node. Initializing... 07/20/2026 01:41:53.234 7f0301ff2f40 [contrail-device-manager] [ERROR]: Cannot write http_port 8096 to /tmp/contrail-device-manager.1.http_port 07/20/2026 01:41:53.234 7f0301ff2f40 [contrail-device-manager] [ERROR]: Starting Introspect on HTTP Port 8096 07/20/2026 01:41:56.254 7f03065db5c0 [contrail-device-manager] [ERROR]: predef payloads file does not exist: /opt/contrail/fabric_ansible_playbooks/conf/predef_payloads.json 07/20/2026 01:41:56.254 7f03065db5c0 [contrail-device-manager] [ERROR]: Unable to load init data 07/20/2026 01:41:56.254 7f03065db5c0 [contrail-device-manager] [WARNING]: VNCCassandra started with driver cql 07/20/2026 01:41:56.254 7f03065db5c0 [contrail-device-manager] [WARNING]: initializing cassandra cluster 07/20/2026 01:41:56.255 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster before create 07/20/2026 01:41:56.261 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster created 07/20/2026 01:41:56.326 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster DB connected 07/20/2026 01:41:56.333 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster init KS 07/20/2026 01:41:56.344 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:56.390 7f03065db5c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace', created 07/20/2026 01:41:56.393 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:56.398 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:56.452 7f03065db5c0 [contrail-device-manager] [WARNING]: keyspace 'dm_keyspace' fixed with replication {'class': 'SimpleStrategy', 'replication_factor': None} 07/20/2026 01:41:56.457 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:56.457 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster KS are ready 07/20/2026 01:41:56.501 7f03065db5c0 [contrail-device-manager] [WARNING]: CassandraDriverCQL.__init__Cluster CF are ready 07/20/2026 01:41:56.513 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:56.620 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table', created 07/20/2026 01:41:56.622 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:56.628 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:56.716 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pr_vn_ip_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:56.719 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:56.725 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:56.846 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table', created 07/20/2026 01:41:56.867 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:56.906 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:57.005 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pr_asn_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:57.012 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:57.022 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:57.267 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table', created 07/20/2026 01:41:57.271 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:57.279 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:57.342 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_ni_ipv6_ll_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:57.345 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:57.349 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:57.637 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table', created 07/20/2026 01:41:57.644 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:57.807 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock acquired. Performing schema change... 07/20/2026 01:41:57.956 7f03065db5c0 [contrail-device-manager] [WARNING]: table 'dm_pnf_resource_table' fixed with properties {'gc_grace_seconds': 864000} 07/20/2026 01:41:57.962 7f03065db5c0 [contrail-device-manager] [WARNING]: Zookeeper lock released. 07/20/2026 01:41:57.994 7f03065db5c0 [contrail-device-manager] [WARNING]: Initializing RabbitMQ connection, urls ['pyamqp://guest:guest@10.0.0.50:5673//'] 07/20/2026 01:41:57.994 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/20/2026 01:41:58.002 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED 07/20/2026 01:41:58.023 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/20/2026 01:41:58.071 7f03065db5c0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/20/2026 01:41:58.083 7f030066a0e0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection down 07/20/2026 01:41:58.105 7f030066a0e0 [contrail-device-manager] [WARNING]: RabbitMQ publish connection ESTABLISHED 07/20/2026 01:41:58.496 7f030066a360 [contrail-device-manager] [ERROR]: Error in rabbitmq drainer greenlet: 'DeviceManagerLogger' object has no attribute 'warn' 07/20/2026 01:41:58.496 7f030066a360 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection down 07/20/2026 01:41:58.524 7f030066a360 [contrail-device-manager] [WARNING]: RabbitMQ drainer connection ESTABLISHED