Started by upstream project "pipeline-templates" build number 67 originally caused by: Triggered by Gerrit: https://gerrit.opensdn.io/c/opensdn-io/tf-analytics/+/1033 Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on slave-openstack-gz1_1 (openstack) in workspace /home/jenkins/workspace/collect-logs-and-cleanup [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is disabled by the job configuration... [WS-CLEANUP] Done The recommended git tool is: git No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-jenkins.git > git init /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision f2e7cec4d3d9ee77eeb3fd101530c908c7a0a470 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f f2e7cec4d3d9ee77eeb3fd101530c908c7a0a470 # timeout=10 Commit message: "Merge "use juju-k8s instead of juju-os"" > git rev-list --no-walk f2e7cec4d3d9ee77eeb3fd101530c908c7a0a470 # timeout=10 The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-devstack.git > git init /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-devstack # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-devstack.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-devstack.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-devstack.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision a70e1b89218f73b6c9f6780530074f429c49b896 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10 Commit message: "tune CONFIG_API_WORKER_COUNT for ansible" > git rev-list --no-walk a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10 Copied 4 artifacts from "pipeline-templates" build number 67 [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins6485530617230921603.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export GERRIT_PIPELINE=templates ++ GERRIT_PIPELINE=templates ++ export GERRIT_URL=https://gerrit.opensdn.io/ ++ GERRIT_URL=https://gerrit.opensdn.io/ ++ export GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export GERRIT_PROJECT=opensdn-io/tf-analytics ++ GERRIT_PROJECT=opensdn-io/tf-analytics ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + desc='Pipeline: pipeline-templates-67 Random: 79243 Stream: juju-k8s-ha' + desc+='
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha' + echo 'DESCRIPTION Pipeline: pipeline-templates-67 Random: 79243 Stream: juju-k8s-ha
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha' DESCRIPTION Pipeline: pipeline-templates-67 Random: 79243 Stream: juju-k8s-ha
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha [description-setter] Description set: Pipeline: pipeline-templates-67 Random: 79243 Stream: juju-k8s-ha
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha' [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins18120711716527447761.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export GERRIT_PIPELINE=templates ++ GERRIT_PIPELINE=templates ++ export GERRIT_URL=https://gerrit.opensdn.io/ ++ GERRIT_URL=https://gerrit.opensdn.io/ ++ export GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export GERRIT_PROJECT=opensdn-io/tf-analytics ++ GERRIT_PROJECT=opensdn-io/tf-analytics ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-jenkins ./patchsets-info.json + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-devstack ./patchsets-info.json [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins17757059574196083249.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-67 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-1_0_3_3-2 ++ export GERRIT_PIPELINE=templates ++ GERRIT_PIPELINE=templates ++ export GERRIT_URL=https://gerrit.opensdn.io/ ++ GERRIT_URL=https://gerrit.opensdn.io/ ++ export GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ GERRIT_CHANGE_ID=Iefc0022eca9527ac70c95cb66ea7edd057270b60 ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export GERRIT_PROJECT=opensdn-io/tf-analytics ++ GERRIT_PROJECT=opensdn-io/tf-analytics ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + source /home/jenkins/workspace/collect-logs-and-cleanup/deps.collect-logs-and-cleanup.79243.env ++ export PROVIDER=openstack ++ PROVIDER=openstack ++ export ENVIRONMENT_OS=ubuntu20 ++ ENVIRONMENT_OS=ubuntu20 ++ export IMAGE=b2b4b2de-c5b4-4181-b613-3d6aa4932ada ++ IMAGE=b2b4b2de-c5b4-4181-b613-3d6aa4932ada ++ export IMAGE_SSH_USER=ubuntu ++ IMAGE_SSH_USER=ubuntu ++ export INSTANCE_IDS=6c556356-26f1-4750-9245-63d05cd140ae,4f7b1361-2559-4918-ba31-8129d44927c9,58999ae9-c33a-4007-8f6b-b20d1e39f115,d4facccf-65e4-4371-8aa2-62f3446b7685,57feafa8-f101-479f-8ba2-321575026b9a,87f2c14d-dcbe-4585-a174-b4385c7ec604, ++ INSTANCE_IDS=6c556356-26f1-4750-9245-63d05cd140ae,4f7b1361-2559-4918-ba31-8129d44927c9,58999ae9-c33a-4007-8f6b-b20d1e39f115,d4facccf-65e4-4371-8aa2-62f3446b7685,57feafa8-f101-479f-8ba2-321575026b9a,87f2c14d-dcbe-4585-a174-b4385c7ec604, ++ export instance_ip=10.0.0.30 ++ instance_ip=10.0.0.30 ++ export JUJU_CONTROLLER_NODES=10.0.0.30, ++ JUJU_CONTROLLER_NODES=10.0.0.30, ++ export CONTROLLER_NODES=10.0.0.33,10.0.0.17,10.0.0.37, ++ CONTROLLER_NODES=10.0.0.33,10.0.0.17,10.0.0.37, ++ export AGENT_NODES=10.0.0.44,10.0.0.24, ++ AGENT_NODES=10.0.0.44,10.0.0.24, ++ export ORCHESTRATOR=kubernetes ++ ORCHESTRATOR=kubernetes ++ export DEPLOYER=juju ++ DEPLOYER=juju ++ export JOB_LOGS_PATH=juju-k8s-ha ++ JOB_LOGS_PATH=juju-k8s-ha + source /home/jenkins/workspace/collect-logs-and-cleanup/vars.collect-logs-and-cleanup.79243.env ++ export MONITORING_DEPLOY_TARGET=HA ++ MONITORING_DEPLOY_TARGET=HA ++ export MONITORING_DEPLOYER=juju ++ MONITORING_DEPLOYER=juju ++ export MONITORING_ORCHESTRATOR=k8s ++ MONITORING_ORCHESTRATOR=k8s + export FULL_LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha + FULL_LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha + [[ -n 10.0.0.30 ]] + /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins/jobs/devstack/juju/collect_logs.sh INFO: wait for host Linux jcn-jenkins-deploy-platform-juju-k8s-670-1 5.4.0-167-generic #184-Ubuntu SMP Tue Oct 31 09:21:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux INFO: collect logs INFO: Deploy juju/logs (collect-logs-and-cleanup) Warning: Permanently added '10.0.0.30' (ECDSA) to the list of known hosts. Warning: Permanently added '10.0.0.30' (ECDSA) to the list of known hosts. INFO: =================== Mon Sep 16 10:19:44 UTC 2024 =================== [there is no tf devenv configuration to load] INFO: Applying stages logs INFO: Running stage logs at Mon Sep 16 10:19:44 UTC 2024 INFO: collecting logs... INFO: machines to ssh: 0 1 2 3 4 INFO: collecting from 0 INFO: Collecting docker logs WARNING: No swap limit support INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/contrail INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-alarm-gen INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-api INFO: Collecting tf logs: /var/log/contrail/analytics-collector INFO: Collecting tf logs: /var/log/contrail/analytics-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-snmp-collector INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-topology INFO: Collecting tf logs: /var/log/contrail/cni INFO: Collecting tf logs: /var/log/contrail/config-api INFO: Collecting tf logs: /var/log/contrail/config-database INFO: Collecting tf logs: /var/log/contrail/config-database-cassandra INFO: Collecting tf logs: /var/log/contrail/config-database-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-database-provisioner INFO: Collecting tf logs: /var/log/contrail/config-database-rabbitmq INFO: Collecting tf logs: /var/log/contrail/config-database-zookeeper INFO: Collecting tf logs: /var/log/contrail/config-device-manager INFO: Collecting tf logs: /var/log/contrail/config-dnsmasq INFO: Collecting tf logs: /var/log/contrail/config-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-provisioner INFO: Collecting tf logs: /var/log/contrail/config-schema INFO: Collecting tf logs: /var/log/contrail/config-svc-monitor INFO: Collecting tf logs: /var/log/contrail/control-control INFO: Collecting tf logs: /var/log/contrail/control-dns INFO: Collecting tf logs: /var/log/contrail/control-named INFO: Collecting tf logs: /var/log/contrail/control-nodemgr INFO: Collecting tf logs: /var/log/contrail/control-provisioner INFO: Collecting tf logs: /var/log/contrail/database INFO: Collecting tf logs: /var/log/contrail/database-cassandra INFO: Collecting tf logs: /var/log/contrail/database-nodemgr INFO: Collecting tf logs: /var/log/contrail/database-provisioner INFO: Collecting tf logs: /var/log/contrail/database-query-engine INFO: Collecting tf logs: /var/log/contrail/kafka INFO: Collecting tf logs: /var/log/contrail/kubernetes-kube-manager INFO: Collecting tf logs: /var/log/contrail/node-init INFO: Collecting tf logs: /var/log/contrail/rabbitmq INFO: Collecting tf logs: /var/log/contrail/webui-job INFO: Collecting tf logs: /var/log/contrail/webui-web INFO: Collecting tf logs: /var/log/contrail/zookeeper INFO: Collecting tf logs: save_introspect_info INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8100/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8101/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8103/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8104/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8112/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8113/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8114/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8083/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8084/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8089/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8090/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8091/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:8092/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:5995/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:5920/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-1:5921/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Sep 16 07:45 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. -rw-r--r-- 1 root root 0 Sep 16 07:44 kdump_lock -rw-r--r-- 1 root root 286 Sep 16 07:57 kexec_cmd INFO: content of /var/crashes /var/crashes: total 8 drwxrwxrwx 2 root root 4096 Sep 16 09:41 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. INFO: Collecting openstack logs INFO: Collecting kubernetes logs INFO: Collecting kubernetes object info INFO: Processing namespace contrail INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/contrail-kubemanager INFO: processing secret/contrail-kubemanager-token-8g8t8 INFO: processing secret/default-token-hf6v8 INFO: collect info for serviceaccounts INFO: processing serviceaccount/contrail-kubemanager INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace default INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/ctest-nginx-svc-93763619 INFO: processing endpoints/kubernetes INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-qdn5m INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: processing service/ctest-nginx-svc-93763619 INFO: processing service/kubernetes INFO: collect info for jobs INFO: Processing namespace kube-node-lease INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-x4r75 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-public INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-28ql4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-system INFO: collect info for pod INFO: processing pod/coredns-6f867cd986-68458 INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: processing deployment.apps/coredns INFO: collect info for replicaset.apps INFO: processing replicaset.apps/coredns-6f867cd986 INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/coredns INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/kube-dns INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/attachdetach-controller-token-69ss9 INFO: processing secret/auth-admin-et8597cyvt INFO: processing secret/auth-admin-w8ztkbjjzi INFO: processing secret/auth-kube-controller-manager-i813ztsj98 INFO: processing secret/auth-kube-controller-manager-y5kqt48fwl INFO: processing secret/auth-kube-proxy-gzwdjb7w36 INFO: processing secret/auth-kubernetes-worker-0-smpnrbmzm3 INFO: processing secret/auth-kubernetes-worker-0-zq251txw67 INFO: processing secret/auth-kubernetes-worker-1-5tfm3l2nhi INFO: processing secret/auth-kubernetes-worker-1-e8h5btohup INFO: processing secret/auth-system-kube-scheduler-1w47qq6g30 INFO: processing secret/auth-system-kube-scheduler-qzpurxutta INFO: processing secret/auth-system-monitoring-1858qc4x6a INFO: processing secret/auth-system-monitoring-l631bmiylz INFO: processing secret/certificate-controller-token-c2z4g INFO: processing secret/clusterrole-aggregation-controller-token-zcc82 INFO: processing secret/coredns-token-snq7c INFO: processing secret/cronjob-controller-token-69wvc INFO: processing secret/daemon-set-controller-token-57sts INFO: processing secret/default-token-czspg INFO: processing secret/deployment-controller-token-6n9xf INFO: processing secret/disruption-controller-token-76wqg INFO: processing secret/endpoint-controller-token-j4vl6 INFO: processing secret/endpointslice-controller-token-lx4wh INFO: processing secret/endpointslicemirroring-controller-token-kxmkf INFO: processing secret/ephemeral-volume-controller-token-bwtzn INFO: processing secret/expand-controller-token-kq5p8 INFO: processing secret/generic-garbage-collector-token-9nwds INFO: processing secret/horizontal-pod-autoscaler-token-m2frb INFO: processing secret/job-controller-token-65lqw INFO: processing secret/namespace-controller-token-9ztbl INFO: processing secret/node-controller-token-k5sgr INFO: processing secret/persistent-volume-binder-token-c64gw INFO: processing secret/pod-garbage-collector-token-v82qp INFO: processing secret/pv-protection-controller-token-fjmms INFO: processing secret/pvc-protection-controller-token-pnd66 INFO: processing secret/replicaset-controller-token-wtt8j INFO: processing secret/replication-controller-token-hvws5 INFO: processing secret/resourcequota-controller-token-8ntnd INFO: processing secret/root-ca-cert-publisher-token-kqz8l INFO: processing secret/service-account-controller-token-2cln9 INFO: processing secret/service-controller-token-hdtvw INFO: processing secret/statefulset-controller-token-jwsfv INFO: processing secret/ttl-after-finished-controller-token-9z2zr INFO: processing secret/ttl-controller-token-6jjd4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/attachdetach-controller INFO: processing serviceaccount/certificate-controller INFO: processing serviceaccount/clusterrole-aggregation-controller INFO: processing serviceaccount/coredns INFO: processing serviceaccount/cronjob-controller INFO: processing serviceaccount/daemon-set-controller INFO: processing serviceaccount/default INFO: processing serviceaccount/deployment-controller INFO: processing serviceaccount/disruption-controller INFO: processing serviceaccount/endpoint-controller INFO: processing serviceaccount/endpointslice-controller INFO: processing serviceaccount/endpointslicemirroring-controller INFO: processing serviceaccount/ephemeral-volume-controller INFO: processing serviceaccount/expand-controller INFO: processing serviceaccount/generic-garbage-collector INFO: processing serviceaccount/horizontal-pod-autoscaler INFO: processing serviceaccount/job-controller INFO: processing serviceaccount/namespace-controller INFO: processing serviceaccount/node-controller INFO: processing serviceaccount/persistent-volume-binder INFO: processing serviceaccount/pod-garbage-collector INFO: processing serviceaccount/pv-protection-controller INFO: processing serviceaccount/pvc-protection-controller INFO: processing serviceaccount/replicaset-controller INFO: processing serviceaccount/replication-controller INFO: processing serviceaccount/resourcequota-controller INFO: processing serviceaccount/root-ca-cert-publisher INFO: processing serviceaccount/service-account-controller INFO: processing serviceaccount/service-controller INFO: processing serviceaccount/statefulset-controller INFO: processing serviceaccount/ttl-after-finished-controller INFO: processing serviceaccount/ttl-controller INFO: collect info for services INFO: processing service/kube-dns INFO: collect info for jobs INFO: collect info for persistentvolumes INFO: collect info for customresourcedefinitions INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io INFO: collect info for storageclasses INFO: info collected INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/0 ~ ~ INFO: collecting from 1 INFO: Collecting docker logs WARNING: No swap limit support INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/contrail INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-alarm-gen INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-api INFO: Collecting tf logs: /var/log/contrail/analytics-collector INFO: Collecting tf logs: /var/log/contrail/analytics-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-snmp-collector INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-topology INFO: Collecting tf logs: /var/log/contrail/cni INFO: Collecting tf logs: /var/log/contrail/config-api INFO: Collecting tf logs: /var/log/contrail/config-database INFO: Collecting tf logs: /var/log/contrail/config-database-cassandra INFO: Collecting tf logs: /var/log/contrail/config-database-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-database-provisioner INFO: Collecting tf logs: /var/log/contrail/config-database-rabbitmq INFO: Collecting tf logs: /var/log/contrail/config-database-zookeeper INFO: Collecting tf logs: /var/log/contrail/config-device-manager INFO: Collecting tf logs: /var/log/contrail/config-dnsmasq INFO: Collecting tf logs: /var/log/contrail/config-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-provisioner INFO: Collecting tf logs: /var/log/contrail/config-schema INFO: Collecting tf logs: /var/log/contrail/config-svc-monitor INFO: Collecting tf logs: /var/log/contrail/control-control INFO: Collecting tf logs: /var/log/contrail/control-dns INFO: Collecting tf logs: /var/log/contrail/control-named INFO: Collecting tf logs: /var/log/contrail/control-nodemgr INFO: Collecting tf logs: /var/log/contrail/control-provisioner INFO: Collecting tf logs: /var/log/contrail/database INFO: Collecting tf logs: /var/log/contrail/database-cassandra INFO: Collecting tf logs: /var/log/contrail/database-nodemgr INFO: Collecting tf logs: /var/log/contrail/database-provisioner INFO: Collecting tf logs: /var/log/contrail/database-query-engine INFO: Collecting tf logs: /var/log/contrail/kafka INFO: Collecting tf logs: /var/log/contrail/kubernetes-kube-manager INFO: Collecting tf logs: /var/log/contrail/node-init INFO: Collecting tf logs: /var/log/contrail/rabbitmq INFO: Collecting tf logs: /var/log/contrail/webui-job INFO: Collecting tf logs: /var/log/contrail/webui-web INFO: Collecting tf logs: /var/log/contrail/zookeeper INFO: Collecting tf logs: save_introspect_info INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8100/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8101/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8103/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8104/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8112/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8113/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8114/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8083/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8084/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8087/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8096/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8089/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8090/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8091/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:8092/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:5995/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:5920/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-2:5921/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Sep 16 07:45 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. -rw-r--r-- 1 root root 0 Sep 16 07:45 kdump_lock -rw-r--r-- 1 root root 286 Sep 16 07:59 kexec_cmd INFO: content of /var/crashes /var/crashes: total 8 drwxrwxrwx 2 root root 4096 Sep 16 09:41 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. INFO: Collecting openstack logs INFO: Collecting kubernetes logs INFO: Collecting kubernetes object info INFO: Processing namespace contrail INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/contrail-kubemanager INFO: processing secret/contrail-kubemanager-token-8g8t8 INFO: processing secret/default-token-hf6v8 INFO: collect info for serviceaccounts INFO: processing serviceaccount/contrail-kubemanager INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace default INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/ctest-nginx-svc-93763619 INFO: processing endpoints/kubernetes INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-qdn5m INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: processing service/ctest-nginx-svc-93763619 INFO: processing service/kubernetes INFO: collect info for jobs INFO: Processing namespace kube-node-lease INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-x4r75 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-public INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-28ql4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-system INFO: collect info for pod INFO: processing pod/coredns-6f867cd986-68458 INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: processing deployment.apps/coredns INFO: collect info for replicaset.apps INFO: processing replicaset.apps/coredns-6f867cd986 INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/coredns INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/kube-dns INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/attachdetach-controller-token-69ss9 INFO: processing secret/auth-admin-et8597cyvt INFO: processing secret/auth-admin-w8ztkbjjzi INFO: processing secret/auth-kube-controller-manager-i813ztsj98 INFO: processing secret/auth-kube-controller-manager-y5kqt48fwl INFO: processing secret/auth-kube-proxy-gzwdjb7w36 INFO: processing secret/auth-kubernetes-worker-0-smpnrbmzm3 INFO: processing secret/auth-kubernetes-worker-0-zq251txw67 INFO: processing secret/auth-kubernetes-worker-1-5tfm3l2nhi INFO: processing secret/auth-kubernetes-worker-1-e8h5btohup INFO: processing secret/auth-system-kube-scheduler-1w47qq6g30 INFO: processing secret/auth-system-kube-scheduler-qzpurxutta INFO: processing secret/auth-system-monitoring-1858qc4x6a INFO: processing secret/auth-system-monitoring-l631bmiylz INFO: processing secret/certificate-controller-token-c2z4g INFO: processing secret/clusterrole-aggregation-controller-token-zcc82 INFO: processing secret/coredns-token-snq7c INFO: processing secret/cronjob-controller-token-69wvc INFO: processing secret/daemon-set-controller-token-57sts INFO: processing secret/default-token-czspg INFO: processing secret/deployment-controller-token-6n9xf INFO: processing secret/disruption-controller-token-76wqg INFO: processing secret/endpoint-controller-token-j4vl6 INFO: processing secret/endpointslice-controller-token-lx4wh INFO: processing secret/endpointslicemirroring-controller-token-kxmkf INFO: processing secret/ephemeral-volume-controller-token-bwtzn INFO: processing secret/expand-controller-token-kq5p8 INFO: processing secret/generic-garbage-collector-token-9nwds INFO: processing secret/horizontal-pod-autoscaler-token-m2frb INFO: processing secret/job-controller-token-65lqw INFO: processing secret/namespace-controller-token-9ztbl INFO: processing secret/node-controller-token-k5sgr INFO: processing secret/persistent-volume-binder-token-c64gw INFO: processing secret/pod-garbage-collector-token-v82qp INFO: processing secret/pv-protection-controller-token-fjmms INFO: processing secret/pvc-protection-controller-token-pnd66 INFO: processing secret/replicaset-controller-token-wtt8j INFO: processing secret/replication-controller-token-hvws5 INFO: processing secret/resourcequota-controller-token-8ntnd INFO: processing secret/root-ca-cert-publisher-token-kqz8l INFO: processing secret/service-account-controller-token-2cln9 INFO: processing secret/service-controller-token-hdtvw INFO: processing secret/statefulset-controller-token-jwsfv INFO: processing secret/ttl-after-finished-controller-token-9z2zr INFO: processing secret/ttl-controller-token-6jjd4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/attachdetach-controller INFO: processing serviceaccount/certificate-controller INFO: processing serviceaccount/clusterrole-aggregation-controller INFO: processing serviceaccount/coredns INFO: processing serviceaccount/cronjob-controller INFO: processing serviceaccount/daemon-set-controller INFO: processing serviceaccount/default INFO: processing serviceaccount/deployment-controller INFO: processing serviceaccount/disruption-controller INFO: processing serviceaccount/endpoint-controller INFO: processing serviceaccount/endpointslice-controller INFO: processing serviceaccount/endpointslicemirroring-controller INFO: processing serviceaccount/ephemeral-volume-controller INFO: processing serviceaccount/expand-controller INFO: processing serviceaccount/generic-garbage-collector INFO: processing serviceaccount/horizontal-pod-autoscaler INFO: processing serviceaccount/job-controller INFO: processing serviceaccount/namespace-controller INFO: processing serviceaccount/node-controller INFO: processing serviceaccount/persistent-volume-binder INFO: processing serviceaccount/pod-garbage-collector INFO: processing serviceaccount/pv-protection-controller INFO: processing serviceaccount/pvc-protection-controller INFO: processing serviceaccount/replicaset-controller INFO: processing serviceaccount/replication-controller INFO: processing serviceaccount/resourcequota-controller INFO: processing serviceaccount/root-ca-cert-publisher INFO: processing serviceaccount/service-account-controller INFO: processing serviceaccount/service-controller INFO: processing serviceaccount/statefulset-controller INFO: processing serviceaccount/ttl-after-finished-controller INFO: processing serviceaccount/ttl-controller INFO: collect info for services INFO: processing service/kube-dns INFO: collect info for jobs INFO: collect info for persistentvolumes INFO: collect info for customresourcedefinitions INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io INFO: collect info for storageclasses INFO: info collected INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/1 ~ ~ INFO: collecting from 2 INFO: Collecting docker logs WARNING: No swap limit support INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/contrail INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-alarm-gen INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-api INFO: Collecting tf logs: /var/log/contrail/analytics-collector INFO: Collecting tf logs: /var/log/contrail/analytics-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-nodemgr INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-provisioner INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-snmp-collector INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-topology INFO: Collecting tf logs: /var/log/contrail/cni INFO: Collecting tf logs: /var/log/contrail/config-api INFO: Collecting tf logs: /var/log/contrail/config-database INFO: Collecting tf logs: /var/log/contrail/config-database-cassandra INFO: Collecting tf logs: /var/log/contrail/config-database-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-database-provisioner INFO: Collecting tf logs: /var/log/contrail/config-database-rabbitmq INFO: Collecting tf logs: /var/log/contrail/config-database-zookeeper INFO: Collecting tf logs: /var/log/contrail/config-device-manager INFO: Collecting tf logs: /var/log/contrail/config-dnsmasq INFO: Collecting tf logs: /var/log/contrail/config-nodemgr INFO: Collecting tf logs: /var/log/contrail/config-provisioner INFO: Collecting tf logs: /var/log/contrail/config-schema INFO: Collecting tf logs: /var/log/contrail/config-svc-monitor INFO: Collecting tf logs: /var/log/contrail/control-control INFO: Collecting tf logs: /var/log/contrail/control-dns INFO: Collecting tf logs: /var/log/contrail/control-named INFO: Collecting tf logs: /var/log/contrail/control-nodemgr INFO: Collecting tf logs: /var/log/contrail/control-provisioner INFO: Collecting tf logs: /var/log/contrail/database INFO: Collecting tf logs: /var/log/contrail/database-cassandra INFO: Collecting tf logs: /var/log/contrail/database-nodemgr INFO: Collecting tf logs: /var/log/contrail/database-provisioner INFO: Collecting tf logs: /var/log/contrail/database-query-engine INFO: Collecting tf logs: /var/log/contrail/kafka INFO: Collecting tf logs: /var/log/contrail/kubernetes-kube-manager INFO: Collecting tf logs: /var/log/contrail/node-init INFO: Collecting tf logs: /var/log/contrail/rabbitmq INFO: Collecting tf logs: /var/log/contrail/webui-job INFO: Collecting tf logs: /var/log/contrail/webui-web INFO: Collecting tf logs: /var/log/contrail/zookeeper INFO: Collecting tf logs: save_introspect_info INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8100/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8101/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8103/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8104/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8108/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8112/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8113/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8114/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8083/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8084/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8088/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8089/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8090/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8091/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:8092/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:5995/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:5920/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-juju-k8s-670-3:5921/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Sep 16 07:46 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. -rw-r--r-- 1 root root 0 Sep 16 07:46 kdump_lock -rw-r--r-- 1 root root 286 Sep 16 08:01 kexec_cmd INFO: content of /var/crashes /var/crashes: total 8 drwxrwxrwx 2 root root 4096 Sep 16 09:41 . drwxr-xr-x 14 root root 4096 Sep 16 09:41 .. INFO: Collecting openstack logs INFO: Collecting kubernetes logs INFO: Collecting kubernetes object info INFO: Processing namespace contrail INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/contrail-kubemanager INFO: processing secret/contrail-kubemanager-token-8g8t8 INFO: processing secret/default-token-hf6v8 INFO: collect info for serviceaccounts INFO: processing serviceaccount/contrail-kubemanager INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace default INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/ctest-nginx-svc-93763619 INFO: processing endpoints/kubernetes INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-qdn5m INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: processing service/ctest-nginx-svc-93763619 INFO: processing service/kubernetes INFO: collect info for jobs INFO: Processing namespace kube-node-lease INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-x4r75 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-public INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-28ql4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-system INFO: collect info for pod INFO: processing pod/coredns-6f867cd986-68458 INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: processing deployment.apps/coredns INFO: collect info for replicaset.apps INFO: processing replicaset.apps/coredns-6f867cd986 INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/coredns INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/kube-dns INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/attachdetach-controller-token-69ss9 INFO: processing secret/auth-admin-et8597cyvt INFO: processing secret/auth-admin-w8ztkbjjzi INFO: processing secret/auth-kube-controller-manager-i813ztsj98 INFO: processing secret/auth-kube-controller-manager-y5kqt48fwl INFO: processing secret/auth-kube-proxy-gzwdjb7w36 INFO: processing secret/auth-kubernetes-worker-0-smpnrbmzm3 INFO: processing secret/auth-kubernetes-worker-0-zq251txw67 INFO: processing secret/auth-kubernetes-worker-1-5tfm3l2nhi INFO: processing secret/auth-kubernetes-worker-1-e8h5btohup INFO: processing secret/auth-system-kube-scheduler-1w47qq6g30 INFO: processing secret/auth-system-kube-scheduler-qzpurxutta INFO: processing secret/auth-system-monitoring-1858qc4x6a INFO: processing secret/auth-system-monitoring-l631bmiylz INFO: processing secret/certificate-controller-token-c2z4g INFO: processing secret/clusterrole-aggregation-controller-token-zcc82 INFO: processing secret/coredns-token-snq7c INFO: processing secret/cronjob-controller-token-69wvc INFO: processing secret/daemon-set-controller-token-57sts INFO: processing secret/default-token-czspg INFO: processing secret/deployment-controller-token-6n9xf INFO: processing secret/disruption-controller-token-76wqg INFO: processing secret/endpoint-controller-token-j4vl6 INFO: processing secret/endpointslice-controller-token-lx4wh INFO: processing secret/endpointslicemirroring-controller-token-kxmkf INFO: processing secret/ephemeral-volume-controller-token-bwtzn INFO: processing secret/expand-controller-token-kq5p8 INFO: processing secret/generic-garbage-collector-token-9nwds INFO: processing secret/horizontal-pod-autoscaler-token-m2frb INFO: processing secret/job-controller-token-65lqw INFO: processing secret/namespace-controller-token-9ztbl INFO: processing secret/node-controller-token-k5sgr INFO: processing secret/persistent-volume-binder-token-c64gw INFO: processing secret/pod-garbage-collector-token-v82qp INFO: processing secret/pv-protection-controller-token-fjmms INFO: processing secret/pvc-protection-controller-token-pnd66 INFO: processing secret/replicaset-controller-token-wtt8j INFO: processing secret/replication-controller-token-hvws5 INFO: processing secret/resourcequota-controller-token-8ntnd INFO: processing secret/root-ca-cert-publisher-token-kqz8l INFO: processing secret/service-account-controller-token-2cln9 INFO: processing secret/service-controller-token-hdtvw INFO: processing secret/statefulset-controller-token-jwsfv INFO: processing secret/ttl-after-finished-controller-token-9z2zr INFO: processing secret/ttl-controller-token-6jjd4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/attachdetach-controller INFO: processing serviceaccount/certificate-controller INFO: processing serviceaccount/clusterrole-aggregation-controller INFO: processing serviceaccount/coredns INFO: processing serviceaccount/cronjob-controller INFO: processing serviceaccount/daemon-set-controller INFO: processing serviceaccount/default INFO: processing serviceaccount/deployment-controller INFO: processing serviceaccount/disruption-controller INFO: processing serviceaccount/endpoint-controller INFO: processing serviceaccount/endpointslice-controller INFO: processing serviceaccount/endpointslicemirroring-controller INFO: processing serviceaccount/ephemeral-volume-controller INFO: processing serviceaccount/expand-controller INFO: processing serviceaccount/generic-garbage-collector INFO: processing serviceaccount/horizontal-pod-autoscaler INFO: processing serviceaccount/job-controller INFO: processing serviceaccount/namespace-controller INFO: processing serviceaccount/node-controller INFO: processing serviceaccount/persistent-volume-binder INFO: processing serviceaccount/pod-garbage-collector INFO: processing serviceaccount/pv-protection-controller INFO: processing serviceaccount/pvc-protection-controller INFO: processing serviceaccount/replicaset-controller INFO: processing serviceaccount/replication-controller INFO: processing serviceaccount/resourcequota-controller INFO: processing serviceaccount/root-ca-cert-publisher INFO: processing serviceaccount/service-account-controller INFO: processing serviceaccount/service-controller INFO: processing serviceaccount/statefulset-controller INFO: processing serviceaccount/ttl-after-finished-controller INFO: processing serviceaccount/ttl-controller INFO: collect info for services INFO: processing service/kube-dns INFO: collect info for jobs INFO: collect info for persistentvolumes INFO: collect info for customresourcedefinitions INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io INFO: collect info for storageclasses INFO: info collected INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/2 ~ ~ INFO: collecting from 3 INFO: Collecting docker logs WARNING: No swap limit support INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/contrail INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: /var/log/contrail/cni INFO: Collecting tf logs: /var/log/contrail/contrail-lbaas-haproxy-stdout.log INFO: Collecting tf logs: /var/log/contrail/node-init INFO: Collecting tf logs: /var/log/contrail/vrouter-agent INFO: Collecting tf logs: /var/log/contrail/vrouter-kernel-build-init INFO: Collecting tf logs: /var/log/contrail/vrouter-nodemgr INFO: Collecting tf logs: /var/log/contrail/vrouter-provisioner INFO: Collecting tf logs: save_introspect_info INFO: Collecting tf logs: introspection request: curl -s http://an-jenkins-deploy-platform-juju-k8s-670-1:8102/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://an-jenkins-deploy-platform-juju-k8s-670-1:8085/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Sep 16 07:51 . drwxr-xr-x 14 root root 4096 Sep 16 09:42 .. -rw-r--r-- 1 root root 0 Sep 16 07:51 kdump_lock -rw-r--r-- 1 root root 286 Sep 16 08:03 kexec_cmd INFO: content of /var/crashes /var/crashes: total 8 drwxr-xr-x 2 root root 4096 Sep 16 09:42 . drwxr-xr-x 14 root root 4096 Sep 16 09:42 .. INFO: Collecting openstack logs INFO: Collecting kubernetes logs INFO: Collecting kubernetes object info INFO: Processing namespace contrail INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/contrail-kubemanager INFO: processing secret/contrail-kubemanager-token-8g8t8 INFO: processing secret/default-token-hf6v8 INFO: collect info for serviceaccounts INFO: processing serviceaccount/contrail-kubemanager INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace default INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/ctest-nginx-svc-93763619 INFO: processing endpoints/kubernetes INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-qdn5m INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: processing service/ctest-nginx-svc-93763619 INFO: processing service/kubernetes INFO: collect info for jobs INFO: Processing namespace kube-node-lease INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-x4r75 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-public INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-28ql4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-system INFO: collect info for pod INFO: processing pod/coredns-6f867cd986-68458 INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: processing deployment.apps/coredns INFO: collect info for replicaset.apps INFO: processing replicaset.apps/coredns-6f867cd986 INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/coredns INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/kube-dns INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/attachdetach-controller-token-69ss9 INFO: processing secret/auth-admin-et8597cyvt INFO: processing secret/auth-admin-w8ztkbjjzi INFO: processing secret/auth-kube-controller-manager-i813ztsj98 INFO: processing secret/auth-kube-controller-manager-y5kqt48fwl INFO: processing secret/auth-kube-proxy-gzwdjb7w36 INFO: processing secret/auth-kubernetes-worker-0-smpnrbmzm3 INFO: processing secret/auth-kubernetes-worker-0-zq251txw67 INFO: processing secret/auth-kubernetes-worker-1-5tfm3l2nhi INFO: processing secret/auth-kubernetes-worker-1-e8h5btohup INFO: processing secret/auth-system-kube-scheduler-1w47qq6g30 INFO: processing secret/auth-system-kube-scheduler-qzpurxutta INFO: processing secret/auth-system-monitoring-1858qc4x6a INFO: processing secret/auth-system-monitoring-l631bmiylz INFO: processing secret/certificate-controller-token-c2z4g INFO: processing secret/clusterrole-aggregation-controller-token-zcc82 INFO: processing secret/coredns-token-snq7c INFO: processing secret/cronjob-controller-token-69wvc INFO: processing secret/daemon-set-controller-token-57sts INFO: processing secret/default-token-czspg INFO: processing secret/deployment-controller-token-6n9xf INFO: processing secret/disruption-controller-token-76wqg INFO: processing secret/endpoint-controller-token-j4vl6 INFO: processing secret/endpointslice-controller-token-lx4wh INFO: processing secret/endpointslicemirroring-controller-token-kxmkf INFO: processing secret/ephemeral-volume-controller-token-bwtzn INFO: processing secret/expand-controller-token-kq5p8 INFO: processing secret/generic-garbage-collector-token-9nwds INFO: processing secret/horizontal-pod-autoscaler-token-m2frb INFO: processing secret/job-controller-token-65lqw INFO: processing secret/namespace-controller-token-9ztbl INFO: processing secret/node-controller-token-k5sgr INFO: processing secret/persistent-volume-binder-token-c64gw INFO: processing secret/pod-garbage-collector-token-v82qp INFO: processing secret/pv-protection-controller-token-fjmms INFO: processing secret/pvc-protection-controller-token-pnd66 INFO: processing secret/replicaset-controller-token-wtt8j INFO: processing secret/replication-controller-token-hvws5 INFO: processing secret/resourcequota-controller-token-8ntnd INFO: processing secret/root-ca-cert-publisher-token-kqz8l INFO: processing secret/service-account-controller-token-2cln9 INFO: processing secret/service-controller-token-hdtvw INFO: processing secret/statefulset-controller-token-jwsfv INFO: processing secret/ttl-after-finished-controller-token-9z2zr INFO: processing secret/ttl-controller-token-6jjd4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/attachdetach-controller INFO: processing serviceaccount/certificate-controller INFO: processing serviceaccount/clusterrole-aggregation-controller INFO: processing serviceaccount/coredns INFO: processing serviceaccount/cronjob-controller INFO: processing serviceaccount/daemon-set-controller INFO: processing serviceaccount/default INFO: processing serviceaccount/deployment-controller INFO: processing serviceaccount/disruption-controller INFO: processing serviceaccount/endpoint-controller INFO: processing serviceaccount/endpointslice-controller INFO: processing serviceaccount/endpointslicemirroring-controller INFO: processing serviceaccount/ephemeral-volume-controller INFO: processing serviceaccount/expand-controller INFO: processing serviceaccount/generic-garbage-collector INFO: processing serviceaccount/horizontal-pod-autoscaler INFO: processing serviceaccount/job-controller INFO: processing serviceaccount/namespace-controller INFO: processing serviceaccount/node-controller INFO: processing serviceaccount/persistent-volume-binder INFO: processing serviceaccount/pod-garbage-collector INFO: processing serviceaccount/pv-protection-controller INFO: processing serviceaccount/pvc-protection-controller INFO: processing serviceaccount/replicaset-controller INFO: processing serviceaccount/replication-controller INFO: processing serviceaccount/resourcequota-controller INFO: processing serviceaccount/root-ca-cert-publisher INFO: processing serviceaccount/service-account-controller INFO: processing serviceaccount/service-controller INFO: processing serviceaccount/statefulset-controller INFO: processing serviceaccount/ttl-after-finished-controller INFO: processing serviceaccount/ttl-controller INFO: collect info for services INFO: processing service/kube-dns INFO: collect info for jobs INFO: collect info for persistentvolumes INFO: collect info for customresourcedefinitions INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io INFO: collect info for storageclasses INFO: info collected INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services INFO: There are no required containers /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/3 ~ ~ INFO: collecting from 4 INFO: Collecting docker logs WARNING: No swap limit support INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/contrail INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: /var/log/contrail/cni INFO: Collecting tf logs: /var/log/contrail/contrail-lbaas-haproxy-stdout.log INFO: Collecting tf logs: /var/log/contrail/node-init INFO: Collecting tf logs: /var/log/contrail/vrouter-agent INFO: Collecting tf logs: /var/log/contrail/vrouter-kernel-build-init INFO: Collecting tf logs: /var/log/contrail/vrouter-nodemgr INFO: Collecting tf logs: /var/log/contrail/vrouter-provisioner INFO: Collecting tf logs: save_introspect_info INFO: Collecting tf logs: introspection request: curl -s http://an-jenkins-deploy-platform-juju-k8s-670-2:8102/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting tf logs: introspection request: curl -s http://an-jenkins-deploy-platform-juju-k8s-670-2:8085/Snh_SandeshUVECacheReq?x=NodeStatus INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Sep 16 07:52 . drwxr-xr-x 14 root root 4096 Sep 16 09:42 .. -rw-r--r-- 1 root root 0 Sep 16 07:52 kdump_lock -rw-r--r-- 1 root root 286 Sep 16 08:06 kexec_cmd INFO: content of /var/crashes /var/crashes: total 8 drwxr-xr-x 2 root root 4096 Sep 16 09:42 . drwxr-xr-x 14 root root 4096 Sep 16 09:42 .. INFO: Collecting openstack logs INFO: Collecting kubernetes logs INFO: Collecting kubernetes object info INFO: Processing namespace contrail INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/contrail-kubemanager INFO: processing secret/contrail-kubemanager-token-8g8t8 INFO: processing secret/default-token-hf6v8 INFO: collect info for serviceaccounts INFO: processing serviceaccount/contrail-kubemanager INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace default INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/ctest-nginx-svc-93763619 INFO: processing endpoints/kubernetes INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-qdn5m INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: processing service/ctest-nginx-svc-93763619 INFO: processing service/kubernetes INFO: collect info for jobs INFO: Processing namespace kube-node-lease INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-x4r75 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-public INFO: collect info for pod INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: collect info for replicaset.apps INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/default-token-28ql4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/default INFO: collect info for services INFO: collect info for jobs INFO: Processing namespace kube-system INFO: collect info for pod INFO: processing pod/coredns-6f867cd986-68458 INFO: collect info for daemonset.apps INFO: collect info for deployment.apps INFO: processing deployment.apps/coredns INFO: collect info for replicaset.apps INFO: processing replicaset.apps/coredns-6f867cd986 INFO: collect info for statefulset.apps INFO: collect info for configmaps INFO: processing configmap/coredns INFO: processing configmap/kube-root-ca.crt INFO: collect info for endpoints INFO: processing endpoints/kube-dns INFO: collect info for persistentvolumeclaims INFO: collect info for secrets INFO: processing secret/attachdetach-controller-token-69ss9 INFO: processing secret/auth-admin-et8597cyvt INFO: processing secret/auth-admin-w8ztkbjjzi INFO: processing secret/auth-kube-controller-manager-i813ztsj98 INFO: processing secret/auth-kube-controller-manager-y5kqt48fwl INFO: processing secret/auth-kube-proxy-gzwdjb7w36 INFO: processing secret/auth-kubernetes-worker-0-smpnrbmzm3 INFO: processing secret/auth-kubernetes-worker-0-zq251txw67 INFO: processing secret/auth-kubernetes-worker-1-5tfm3l2nhi INFO: processing secret/auth-kubernetes-worker-1-e8h5btohup INFO: processing secret/auth-system-kube-scheduler-1w47qq6g30 INFO: processing secret/auth-system-kube-scheduler-qzpurxutta INFO: processing secret/auth-system-monitoring-1858qc4x6a INFO: processing secret/auth-system-monitoring-l631bmiylz INFO: processing secret/certificate-controller-token-c2z4g INFO: processing secret/clusterrole-aggregation-controller-token-zcc82 INFO: processing secret/coredns-token-snq7c INFO: processing secret/cronjob-controller-token-69wvc INFO: processing secret/daemon-set-controller-token-57sts INFO: processing secret/default-token-czspg INFO: processing secret/deployment-controller-token-6n9xf INFO: processing secret/disruption-controller-token-76wqg INFO: processing secret/endpoint-controller-token-j4vl6 INFO: processing secret/endpointslice-controller-token-lx4wh INFO: processing secret/endpointslicemirroring-controller-token-kxmkf INFO: processing secret/ephemeral-volume-controller-token-bwtzn INFO: processing secret/expand-controller-token-kq5p8 INFO: processing secret/generic-garbage-collector-token-9nwds INFO: processing secret/horizontal-pod-autoscaler-token-m2frb INFO: processing secret/job-controller-token-65lqw INFO: processing secret/namespace-controller-token-9ztbl INFO: processing secret/node-controller-token-k5sgr INFO: processing secret/persistent-volume-binder-token-c64gw INFO: processing secret/pod-garbage-collector-token-v82qp INFO: processing secret/pv-protection-controller-token-fjmms INFO: processing secret/pvc-protection-controller-token-pnd66 INFO: processing secret/replicaset-controller-token-wtt8j INFO: processing secret/replication-controller-token-hvws5 INFO: processing secret/resourcequota-controller-token-8ntnd INFO: processing secret/root-ca-cert-publisher-token-kqz8l INFO: processing secret/service-account-controller-token-2cln9 INFO: processing secret/service-controller-token-hdtvw INFO: processing secret/statefulset-controller-token-jwsfv INFO: processing secret/ttl-after-finished-controller-token-9z2zr INFO: processing secret/ttl-controller-token-6jjd4 INFO: collect info for serviceaccounts INFO: processing serviceaccount/attachdetach-controller INFO: processing serviceaccount/certificate-controller INFO: processing serviceaccount/clusterrole-aggregation-controller INFO: processing serviceaccount/coredns INFO: processing serviceaccount/cronjob-controller INFO: processing serviceaccount/daemon-set-controller INFO: processing serviceaccount/default INFO: processing serviceaccount/deployment-controller INFO: processing serviceaccount/disruption-controller INFO: processing serviceaccount/endpoint-controller INFO: processing serviceaccount/endpointslice-controller INFO: processing serviceaccount/endpointslicemirroring-controller INFO: processing serviceaccount/ephemeral-volume-controller INFO: processing serviceaccount/expand-controller INFO: processing serviceaccount/generic-garbage-collector INFO: processing serviceaccount/horizontal-pod-autoscaler INFO: processing serviceaccount/job-controller INFO: processing serviceaccount/namespace-controller INFO: processing serviceaccount/node-controller INFO: processing serviceaccount/persistent-volume-binder INFO: processing serviceaccount/pod-garbage-collector INFO: processing serviceaccount/pv-protection-controller INFO: processing serviceaccount/pvc-protection-controller INFO: processing serviceaccount/replicaset-controller INFO: processing serviceaccount/replication-controller INFO: processing serviceaccount/resourcequota-controller INFO: processing serviceaccount/root-ca-cert-publisher INFO: processing serviceaccount/service-account-controller INFO: processing serviceaccount/service-controller INFO: processing serviceaccount/statefulset-controller INFO: processing serviceaccount/ttl-after-finished-controller INFO: processing serviceaccount/ttl-controller INFO: collect info for services INFO: processing service/kube-dns INFO: collect info for jobs INFO: collect info for persistentvolumes INFO: collect info for customresourcedefinitions INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io INFO: collect info for storageclasses INFO: info collected INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services INFO: There are no required containers /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/4 ~ ~ INFO: Collected juju status INFO: Save juju statuses to logs INFO: Save current juju configuration to logs INFO: Save unit statuses to logs INFO: Stage logs was run successfully Mon Sep 16 10:23:37 UTC 2024 [update tf stack configuration] INFO: collect deployment env INFO: controller_nodes: 10.0.0.17 10.0.0.33 10.0.0.37 INFO: agent_nodes: 10.0.0.24 10.0.0.44 ERROR application "keystone" not found (not found) ERROR application "keystone" not found (not found) couldn't queue one action: "unit-keystone-0" not found ERROR no actions were successfully enqueued, aborting ERROR application "keystone" not found (not found) tf setup profile /home/ubuntu/.tf/stack.env DEPLOYER=juju CONTRAIL_CONTAINER_TAG=master-1_0_3_3-2 CONTRAIL_DEPLOYER_CONTAINER_TAG=master-1_0_3_3-2 CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101 ORCHESTRATOR=kubernetes OPENSTACK_VERSION="ussuri" CONTROLLER_NODES="10.0.0.17 10.0.0.33 10.0.0.37" AGENT_NODES="10.0.0.24 10.0.0.44" CONTROL_NODES="10.0.0.33 10.0.0.17 10.0.0.37 " SSL_ENABLE="false" LEGACY_ANALYTICS_ENABLE="true" HUGE_PAGES_1G= CONTAINER_RUNTIME=docker K8S_CA= DEPLOY_IPA_SERVER= IPA_PASSWORD= AUTH_PORT='35357' ENABLE_NAGIOS='false' SSH_USER='ubuntu' AUTH_URL='' AUTH_PASSWORD='password' DPDK_AGENT_NODES='' AUTH_DOMAIN='admin_domain' CONTROL_NODES='10.0.0.17,10.0.0.33,10.0.0.37' INFO: Successful deployment Mon Sep 16 10:23:41 UTC 2024 DEBUG: kill running child jobs: INFO: Deploy logs finished INFO: Copy logs from host to workspace Warning: Permanently added '10.0.0.30' (ECDSA) to the list of known hosts. ~/workspace/collect-logs-and-cleanup ~/workspace/collect-logs-and-cleanup Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. INFO: Logs collected at http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/33/1033/2/templates_67/juju-k8s-ha ~/workspace/collect-logs-and-cleanup + /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins/jobs/devstack/juju/remove_workers.sh nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:43:02.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.30 | | config_drive | | | created | 2024-09-16T07:41:41Z | | flavor | STD3-1-4 (b2c0fbeb-fafb-4394-adc5-3a9709e76249) | | hostId | b999e0f7b763f0a24e914eab7cd7011ab28fca9818a4788b2ff77a93 | | id | 6c556356-26f1-4750-9245-63d05cd140ae | | image | N/A (booted from volume) | | key_name | worker | | name | jcn_jenkins-deploy-platform-juju-k8s-670_1 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:43:02Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='3cda3be0-8161-4269-a4d9-e01de7c83a72' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:44:47.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.33 | | config_drive | | | created | 2024-09-16T07:44:24Z | | flavor | STD3-4-16 (b6760da7-d124-45c7-a0a7-64f40acae087) | | hostId | 956505f88c2a439f6e53a06e2f6bf66ef4c904bc2f70d1b8c1177972 | | id | 4f7b1361-2559-4918-ba31-8129d44927c9 | | image | N/A (booted from volume) | | key_name | worker | | name | cn_jenkins-deploy-platform-juju-k8s-670_1 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:44:47Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='e0d7c073-3e78-49e8-8de1-22bcffdfa205' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:45:40.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.17 | | config_drive | | | created | 2024-09-16T07:45:17Z | | flavor | STD3-4-16 (b6760da7-d124-45c7-a0a7-64f40acae087) | | hostId | b999e0f7b763f0a24e914eab7cd7011ab28fca9818a4788b2ff77a93 | | id | 58999ae9-c33a-4007-8f6b-b20d1e39f115 | | image | N/A (booted from volume) | | key_name | worker | | name | cn_jenkins-deploy-platform-juju-k8s-670_2 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:45:40Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='f0df9038-ddb9-4c3a-8b1c-26453f3df1ca' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:46:33.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.37 | | config_drive | | | created | 2024-09-16T07:46:10Z | | flavor | STD3-4-16 (b6760da7-d124-45c7-a0a7-64f40acae087) | | hostId | 025076243e91ab295530a7b823370a69ae350befffa56b80df3f0d4f | | id | d4facccf-65e4-4371-8aa2-62f3446b7685 | | image | N/A (booted from volume) | | key_name | worker | | name | cn_jenkins-deploy-platform-juju-k8s-670_3 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:46:33Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='f57aebb0-ef33-4a33-95a7-1daab30a50e1' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:50:58.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.44 | | config_drive | | | created | 2024-09-16T07:49:09Z | | flavor | STD3-2-8 (2d9866a9-e955-4986-b00a-340ca54b2cac) | | hostId | 01d5ccc13dbda12de2194615c55ff6409fe4541263a9c70922652d64 | | id | 57feafa8-f101-479f-8ba2-321575026b9a | | image | N/A (booted from volume) | | key_name | worker | | name | an_jenkins-deploy-platform-juju-k8s-670_1 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:50:58Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='ba81a0ed-477f-478b-8fb7-b8ea3fef2a4c' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-09-16T07:51:55.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.24 | | config_drive | | | created | 2024-09-16T07:51:28Z | | flavor | STD3-2-8 (2d9866a9-e955-4986-b00a-340ca54b2cac) | | hostId | b23ef3b8a67c8736b60f0df8e5945f222abb6988a5a1b3fed4e68fe7 | | id | 87f2c14d-dcbe-4585-a174-b4385c7ec604 | | image | N/A (booted from volume) | | key_name | worker | | name | an_jenkins-deploy-platform-juju-k8s-670_2 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-09-16T07:51:56Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='7f3f7296-f3c0-4c38-a609-2a1a05a64362' | +-----------------------------+----------------------------------------------------------+ Archiving artifacts Finished: SUCCESS