Started by upstream project "pipeline-nightly" build number 26 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on slave-openstack-gz1_1 (openstack) in workspace /home/jenkins/workspace/collect-logs-and-cleanup [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is disabled by the job configuration... [WS-CLEANUP] Done The recommended git tool is: git No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-jenkins.git > git init /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision a254c0f0da78c12e0a8ee66ae1dc5418a1d1c014 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f a254c0f0da78c12e0a8ee66ae1dc5418a1d1c014 # timeout=10 Commit message: "Merge "add tf- images"" > git rev-list --no-walk a254c0f0da78c12e0a8ee66ae1dc5418a1d1c014 # timeout=10 The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/opensdn-io/tf-devstack.git > git init /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-devstack # timeout=10 Fetching upstream changes from https://github.com/opensdn-io/tf-devstack.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-devstack.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/opensdn-io/tf-devstack.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision d9a1892692e9132932141ac1f6602965341ed565 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f d9a1892692e9132932141ac1f6602965341ed565 # timeout=10 Commit message: "Merge "add description of scale testing"" > git rev-list --no-walk d9a1892692e9132932141ac1f6602965341ed565 # timeout=10 Copied 3 artifacts from "pipeline-nightly" build number 26 [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins12137698578227536577.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + desc='Pipeline: pipeline-nightly-26 Random: 99698 Stream: juju-k8s' + desc+='
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26/juju-k8s' + echo 'DESCRIPTION Pipeline: pipeline-nightly-26 Random: 99698 Stream: juju-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26/juju-k8s' DESCRIPTION Pipeline: pipeline-nightly-26 Random: 99698 Stream: juju-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26/juju-k8s [description-setter] Description set: Pipeline: pipeline-nightly-26 Random: 99698 Stream: juju-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26/juju-k8s' [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins8119747636272755012.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-jenkins ./patchsets-info.json + ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-devstack ./patchsets-info.json [collect-logs-and-cleanup] $ /bin/bash -xe /tmp/jenkins17367867547024924855.sh + source /home/jenkins/workspace/collect-logs-and-cleanup/global.env ++ export PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ PIPELINE_BUILD_TAG=jenkins-pipeline-nightly-26 ++ export SLAVE=openstack ++ SLAVE=openstack ++ export SLAVE_REGION=gz1 ++ SLAVE_REGION=gz1 ++ export LOGS_HOST=nexus.gz1.opensdn.io ++ LOGS_HOST=nexus.gz1.opensdn.io ++ export LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26 ++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26 ++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository ++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG=nightly ++ CONTRAIL_CONTAINER_TAG=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly ++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5102 ++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_CONTAINER_TAG_ORIGINAL=nightly ++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=nightly ++ export GERRIT_PIPELINE=nightly ++ GERRIT_PIPELINE=nightly ++ export GERRIT_BRANCH=master ++ GERRIT_BRANCH=master ++ export REPOS_CHANNEL=latest ++ REPOS_CHANNEL=latest + source /home/jenkins/workspace/collect-logs-and-cleanup/deps.collect-logs-and-cleanup.99698.env ++ export PROVIDER=openstack ++ PROVIDER=openstack ++ export ENVIRONMENT_OS=ubuntu20 ++ ENVIRONMENT_OS=ubuntu20 ++ export IMAGE=b2b4b2de-c5b4-4181-b613-3d6aa4932ada ++ IMAGE=b2b4b2de-c5b4-4181-b613-3d6aa4932ada ++ export IMAGE_SSH_USER=ubuntu ++ IMAGE_SSH_USER=ubuntu ++ export INSTANCE_IDS=e7bd8843-736d-46c1-a189-361c23832714,ef707e8b-af2b-47d3-9bf1-4174eb24a593, ++ INSTANCE_IDS=e7bd8843-736d-46c1-a189-361c23832714,ef707e8b-af2b-47d3-9bf1-4174eb24a593, ++ export instance_ip=10.0.0.41 ++ instance_ip=10.0.0.41 ++ export CONTROLLER_NODES=10.0.0.41, ++ CONTROLLER_NODES=10.0.0.41, ++ export AGENT_NODES=10.0.0.27, ++ AGENT_NODES=10.0.0.27, ++ export ORCHESTRATOR=kubernetes ++ ORCHESTRATOR=kubernetes ++ export DEPLOYER=juju ++ DEPLOYER=juju ++ export JOB_LOGS_PATH=juju-k8s ++ JOB_LOGS_PATH=juju-k8s + source /home/jenkins/workspace/collect-logs-and-cleanup/vars.collect-logs-and-cleanup.99698.env ++ export MONITORING_DEPLOY_TARGET=AIO ++ MONITORING_DEPLOY_TARGET=AIO ++ export MONITORING_DEPLOYER=juju ++ MONITORING_DEPLOYER=juju ++ export MONITORING_ORCHESTRATOR=k8s ++ MONITORING_ORCHESTRATOR=k8s + export FULL_LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26/juju-k8s + FULL_LOGS_PATH=/var/www/logs/jenkins_logs/nightly/pipeline_26/juju-k8s + [[ -n 10.0.0.41 ]] + /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins/jobs/devstack/juju/collect_logs.sh INFO: wait for host Linux cn-jenkins-deploy-platform-juju-k8s-757-1 5.4.0-167-generic #184-Ubuntu SMP Tue Oct 31 09:21:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux INFO: collect logs INFO: Deploy juju/logs (collect-logs-and-cleanup) Warning: Permanently added '10.0.0.41' (ECDSA) to the list of known hosts. Warning: Permanently added '10.0.0.41' (ECDSA) to the list of known hosts. INFO: =================== Mon Nov 4 02:02:37 UTC 2024 =================== [there is no tf devenv configuration to load] INFO: Applying stages logs INFO: Running stage logs at Mon Nov 4 02:02:37 UTC 2024 INFO: collecting logs... INFO: machines to ssh: 0 1 INFO: collecting from 0 INFO: Collecting docker logs INFO: There is no any docker installed INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: save_introspect_info INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Nov 4 00:03 . drwxr-xr-x 13 root root 4096 Nov 30 2023 .. -rw-r--r-- 1 root root 0 Nov 4 00:03 kdump_lock -rw-r--r-- 1 root root 286 Nov 4 00:07 kexec_cmd INFO: content of /var/crashes ls: cannot access '/var/crashes': No such file or directory ls: cannot access '/var/crashes': No such file or directory INFO: Collecting openstack logs INFO: Collecting kubernetes logs The connection to the server 10.0.0.41:6443 was refused - did you specify the right host or port? INFO: kubectl is not configured here INFO: Collecting kubernetes object info The connection to the server 10.0.0.41:6443 was refused - did you specify the right host or port? INFO: kubectl is not configured here INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services INFO: There is no any docker installed /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/0 ~ ~ INFO: collecting from 1 INFO: Collecting docker logs INFO: There is no any docker installed INFO: Collecting juju logs INFO: Collecting contrail-status INFO: Collecting system statistics for logs INFO: Collecting TF logs INFO: Collecting tf logs: /etc/cni INFO: Collecting tf logs: save_introspect_info INFO: Collecting core dumps INFO: content of /var/crash /var/crash: total 12 drwxrwxrwt 2 root root 4096 Nov 4 00:05 . drwxr-xr-x 13 root root 4096 Nov 30 2023 .. -rw-r--r-- 1 root root 0 Nov 4 00:05 kdump_lock -rw-r--r-- 1 root root 286 Nov 4 00:10 kexec_cmd INFO: content of /var/crashes ls: cannot access '/var/crashes': No such file or directory ls: cannot access '/var/crashes': No such file or directory INFO: Collecting openstack logs INFO: Collecting kubernetes logs The connection to the server localhost:8080 was refused - did you specify the right host or port? INFO: kubectl is not configured here INFO: Collecting kubernetes object info The connection to the server localhost:8080 was refused - did you specify the right host or port? INFO: kubectl is not configured here INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services INFO: There is no any docker installed /tmp/juju-logs/logs /tmp/juju-logs /tmp/juju-logs ~/.tf/logs/1 ~ ~ INFO: Collected juju status INFO: Save juju statuses to logs INFO: Save current juju configuration to logs INFO: Save unit statuses to logs INFO: Stage logs was run successfully Mon Nov 4 02:02:47 UTC 2024 [update tf stack configuration] INFO: collect deployment env INFO: controller_nodes: INFO: agent_nodes: ERROR could not determine leader for "tf-controller" ERROR application "keystone" not found (not found) ERROR application "keystone" not found (not found) couldn't queue one action: "unit-keystone-0" not found ERROR no actions were successfully enqueued, aborting ERROR application "keystone" not found (not found) tf setup profile /home/ubuntu/.tf/stack.env DEPLOYER=juju CONTRAIL_CONTAINER_TAG=nightly CONTRAIL_DEPLOYER_CONTAINER_TAG=nightly CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5102 ORCHESTRATOR=kubernetes OPENSTACK_VERSION="ussuri" CONTROLLER_NODES="" AGENT_NODES="" CONTROL_NODES="10.0.0.41 " SSL_ENABLE="false" LEGACY_ANALYTICS_ENABLE="true" HUGE_PAGES_1G= CONTAINER_RUNTIME=docker K8S_CA= DEPLOY_IPA_SERVER= IPA_PASSWORD= AUTH_PORT='35357' ENABLE_NAGIOS='false' SSH_USER='ubuntu' AUTH_URL='' AUTH_PASSWORD='password' DPDK_AGENT_NODES='' AUTH_DOMAIN='admin_domain' CONTROL_NODES='' INFO: Successful deployment Mon Nov 4 02:02:48 UTC 2024 DEBUG: kill running child jobs: INFO: Deploy logs finished INFO: Copy logs from host to workspace Warning: Permanently added '10.0.0.41' (ECDSA) to the list of known hosts. ~/workspace/collect-logs-and-cleanup ~/workspace/collect-logs-and-cleanup Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. INFO: Logs collected at http://nexus.gz1.opensdn.io:8082/jenkins_logs/nightly/pipeline_26/juju-k8s ~/workspace/collect-logs-and-cleanup + /home/jenkins/workspace/collect-logs-and-cleanup/src/opensdn-io/tf-jenkins/jobs/devstack/juju/remove_workers.sh nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-11-04T00:03:02.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.41 | | config_drive | | | created | 2024-11-04T00:02:21Z | | flavor | STD3-4-16 (b6760da7-d124-45c7-a0a7-64f40acae087) | | hostId | e2e992a47d054b4c6b978321763a8f88170688c6e78a78201d4fd806 | | id | e7bd8843-736d-46c1-a189-361c23832714 | | image | N/A (booted from volume) | | key_name | worker | | name | cn_jenkins-deploy-platform-juju-k8s-757_1 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-11-04T00:03:02Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='11bb7e06-46e0-4d09-a8e4-4db72b7807dc' | +-----------------------------+----------------------------------------------------------+ nova CLI is deprecated and will be removed in a future release | locked | False | INFO: do down nodes +-----------------------------+----------------------------------------------------------+ | Field | Value | +-----------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | GZ1 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2024-11-04T00:05:10.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | management=10.0.0.27 | | config_drive | | | created | 2024-11-04T00:04:35Z | | flavor | STD3-2-8 (2d9866a9-e955-4986-b00a-340ca54b2cac) | | hostId | 516744180c3a5072d16b36d28f93d6f4702c7c84a8f4f903b2b6e980 | | id | ef707e8b-af2b-47d3-9bf1-4174eb24a593 | | image | N/A (booted from volume) | | key_name | worker | | name | an_jenkins-deploy-platform-juju-k8s-757_1 | | progress | 0 | | project_id | **** | | properties | | | status | ACTIVE | | updated | 2024-11-04T00:05:13Z | | user_id | 25d930adb802408885c71e341481886e | | volumes_attached | id='8cfaa1c6-ae8f-40a0-bfde-3a8739585a99' | +-----------------------------+----------------------------------------------------------+ Archiving artifacts Finished: SUCCESS