Started by upstream project "pipeline-templates" build number 27
originally caused by:
Triggered by Gerrit: https://gerrit.opensdn.io/c/opensdn-io/tf-controller/+/881
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on slave-openstack-gz1_1 (openstack) in workspace /home/jenkins/workspace/collect-logs-and-cleanup@3
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is disabled by the job configuration...
[WS-CLEANUP] Done
The recommended git tool is: git
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/opensdn-io/tf-jenkins.git
> git init /home/jenkins/workspace/collect-logs-and-cleanup@3/src/opensdn-io/tf-jenkins # timeout=10
Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git
> git --version # timeout=10
> git --version # 'git version 2.25.1'
> git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 25a1faddd20785dcb3c0e21a4bb0a3a1ef0b1c0d (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 25a1faddd20785dcb3c0e21a4bb0a3a1ef0b1c0d # timeout=10
Commit message: "change ssh host in merger_monitor to opensdn"
> git rev-list --no-walk 25a1faddd20785dcb3c0e21a4bb0a3a1ef0b1c0d # timeout=10
The recommended git tool is: NONE
No credentials specified
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository https://github.com/opensdn-io/tf-devstack.git
> git init /home/jenkins/workspace/collect-logs-and-cleanup@3/src/opensdn-io/tf-devstack # timeout=10
Fetching upstream changes from https://github.com/opensdn-io/tf-devstack.git
> git --version # timeout=10
> git --version # 'git version 2.25.1'
> git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-devstack.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/opensdn-io/tf-devstack.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision a70e1b89218f73b6c9f6780530074f429c49b896 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10
Commit message: "tune CONFIG_API_WORKER_COUNT for ansible"
> git rev-list --no-walk a70e1b89218f73b6c9f6780530074f429c49b896 # timeout=10
Copied 4 artifacts from "pipeline-templates" build number 27
[collect-logs-and-cleanup@3] $ /bin/bash -xe /tmp/jenkins13012422205013482864.sh
+ source /home/jenkins/workspace/collect-logs-and-cleanup@3/global.env
++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ export SLAVE=openstack
++ SLAVE=openstack
++ export SLAVE_REGION=gz1
++ SLAVE_REGION=gz1
++ export LOGS_HOST=nexus.gz1.opensdn.io
++ LOGS_HOST=nexus.gz1.opensdn.io
++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export GERRIT_PIPELINE=templates
++ GERRIT_PIPELINE=templates
++ export GERRIT_URL=https://gerrit.opensdn.io/
++ GERRIT_URL=https://gerrit.opensdn.io/
++ export GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ export GERRIT_BRANCH=master
++ GERRIT_BRANCH=master
++ export GERRIT_PROJECT=opensdn-io/tf-controller
++ GERRIT_PROJECT=opensdn-io/tf-controller
++ export REPOS_CHANNEL=latest
++ REPOS_CHANNEL=latest
+ desc='Pipeline: pipeline-templates-27 Random: 89232 Stream: ansible-k8s'
+ desc+='
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s'
+ echo 'DESCRIPTION Pipeline: pipeline-templates-27 Random: 89232 Stream: ansible-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s'
DESCRIPTION Pipeline: pipeline-templates-27 Random: 89232 Stream: ansible-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s
[description-setter] Description set: Pipeline: pipeline-templates-27 Random: 89232 Stream: ansible-k8s
Job logs: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s'
[collect-logs-and-cleanup@3] $ /bin/bash -xe /tmp/jenkins16971452033004751102.sh
+ source /home/jenkins/workspace/collect-logs-and-cleanup@3/global.env
++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ export SLAVE=openstack
++ SLAVE=openstack
++ export SLAVE_REGION=gz1
++ SLAVE_REGION=gz1
++ export LOGS_HOST=nexus.gz1.opensdn.io
++ LOGS_HOST=nexus.gz1.opensdn.io
++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export GERRIT_PIPELINE=templates
++ GERRIT_PIPELINE=templates
++ export GERRIT_URL=https://gerrit.opensdn.io/
++ GERRIT_URL=https://gerrit.opensdn.io/
++ export GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ export GERRIT_BRANCH=master
++ GERRIT_BRANCH=master
++ export GERRIT_PROJECT=opensdn-io/tf-controller
++ GERRIT_PROJECT=opensdn-io/tf-controller
++ export REPOS_CHANNEL=latest
++ REPOS_CHANNEL=latest
+ ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-jenkins ./patchsets-info.json
+ ./src/opensdn-io/tf-jenkins/infra/gerrit/apply_patchsets.sh ./src opensdn-io/tf-devstack ./patchsets-info.json
[collect-logs-and-cleanup@3] $ /bin/bash -xe /tmp/jenkins8167892619086887611.sh
+ source /home/jenkins/workspace/collect-logs-and-cleanup@3/global.env
++ export PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ PIPELINE_BUILD_TAG=jenkins-pipeline-templates-27
++ export SLAVE=openstack
++ SLAVE=openstack
++ export SLAVE_REGION=gz1
++ SLAVE_REGION=gz1
++ export LOGS_HOST=nexus.gz1.opensdn.io
++ LOGS_HOST=nexus.gz1.opensdn.io
++ export LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27
++ export LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ LOGS_URL=http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27
++ export SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ SITE_MIRROR=http://nexus.gz1.opensdn.io/repository
++ export CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
++ export CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ DEPLOYER_CONTAINER_REGISTRY_ORIGINAL=nexus.gz1.opensdn.io:5101
++ export CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ CONTRAIL_DEPLOYER_CONTAINER_TAG_ORIGINAL=master-8_8_1-4
++ export GERRIT_PIPELINE=templates
++ GERRIT_PIPELINE=templates
++ export GERRIT_URL=https://gerrit.opensdn.io/
++ GERRIT_URL=https://gerrit.opensdn.io/
++ export GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ GERRIT_CHANGE_ID=Ida330a2d171d279303a008ca5735d1c6d8f08854
++ export GERRIT_BRANCH=master
++ GERRIT_BRANCH=master
++ export GERRIT_PROJECT=opensdn-io/tf-controller
++ GERRIT_PROJECT=opensdn-io/tf-controller
++ export REPOS_CHANNEL=latest
++ REPOS_CHANNEL=latest
+ source /home/jenkins/workspace/collect-logs-and-cleanup@3/deps.collect-logs-and-cleanup.89232.env
++ export PROVIDER=openstack
++ PROVIDER=openstack
++ export ENVIRONMENT_OS=rocky9
++ ENVIRONMENT_OS=rocky9
++ export IMAGE=aa1bd152-a8d4-49ef-85ea-82a4dab8cf7f
++ IMAGE=aa1bd152-a8d4-49ef-85ea-82a4dab8cf7f
++ export IMAGE_SSH_USER=rocky
++ IMAGE_SSH_USER=rocky
++ export INSTANCE_IDS=0101ee00-2606-40a8-9c60-3363fe6e9b68,
++ INSTANCE_IDS=0101ee00-2606-40a8-9c60-3363fe6e9b68,
++ export instance_ip=10.0.0.31
++ instance_ip=10.0.0.31
++ export CONTROLLER_NODES=10.0.0.31,
++ CONTROLLER_NODES=10.0.0.31,
++ export ORCHESTRATOR=kubernetes
++ ORCHESTRATOR=kubernetes
++ export DEPLOYER=ansible
++ DEPLOYER=ansible
++ export JOB_LOGS_PATH=ansible-k8s
++ JOB_LOGS_PATH=ansible-k8s
+ source /home/jenkins/workspace/collect-logs-and-cleanup@3/vars.collect-logs-and-cleanup.89232.env
++ export MONITORING_DEPLOY_TARGET=AIO
++ MONITORING_DEPLOY_TARGET=AIO
++ export MONITORING_DEPLOYER=ansible
++ MONITORING_DEPLOYER=ansible
++ export MONITORING_ORCHESTRATOR=k8s
++ MONITORING_ORCHESTRATOR=k8s
++ export ENVIRONMENT_OS=rocky9
++ ENVIRONMENT_OS=rocky9
+ export FULL_LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s
+ FULL_LOGS_PATH=/var/www/logs/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s
+ [[ -n 10.0.0.31 ]]
+ /home/jenkins/workspace/collect-logs-and-cleanup@3/src/opensdn-io/tf-jenkins/jobs/devstack/ansible/collect_logs.sh
INFO: wait for host
Linux cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 17:09:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
INFO: collect logs
INFO: Deploy ansible/logs (collect-logs-and-cleanup)
Warning: Permanently added '10.0.0.31' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.0.0.31' (ECDSA) to the list of known hosts.
INFO: =================== Mon Aug 26 13:21:40 UTC 2024 ===================
[there is no tf devenv configuration to load]
INFO: Applying stages logs
INFO: Running stage logs at Mon Aug 26 13:21:40 UTC 2024
INFO: collecting logs...
cp: cannot stat '/home/rocky/.tf/*.yml': No such file or directory
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
INFO: Collecting contrail-status
INFO: Collecting system statistics for logs
INFO: Collecting docker logs
INFO: Collecting TF logs
INFO: Collecting tf logs: /etc/contrail
INFO: Collecting tf logs: /etc/cni
INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-alarm-gen
INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-nodemgr
INFO: Collecting tf logs: /var/log/contrail/analytics-alarm-provisioner
INFO: Collecting tf logs: /var/log/contrail/analytics-api
INFO: Collecting tf logs: /var/log/contrail/analytics-collector
INFO: Collecting tf logs: /var/log/contrail/analytics-nodemgr
INFO: Collecting tf logs: /var/log/contrail/analytics-provisioner
INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-nodemgr
INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-provisioner
INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-snmp-collector
INFO: Collecting tf logs: /var/log/contrail/analytics-snmp-topology
INFO: Collecting tf logs: /var/log/contrail/cni
INFO: Collecting tf logs: /var/log/contrail/config-api
INFO: Collecting tf logs: /var/log/contrail/config-database
INFO: Collecting tf logs: /var/log/contrail/config-database-nodemgr
INFO: Collecting tf logs: /var/log/contrail/config-database-provisioner
INFO: Collecting tf logs: /var/log/contrail/config-database-rabbitmq
INFO: Collecting tf logs: /var/log/contrail/config-device-manager
INFO: Collecting tf logs: /var/log/contrail/config-nodemgr
INFO: Collecting tf logs: /var/log/contrail/config-provisioner
INFO: Collecting tf logs: /var/log/contrail/config-schema
INFO: Collecting tf logs: /var/log/contrail/config-svc-monitor
INFO: Collecting tf logs: /var/log/contrail/contrail-lbaas-haproxy-stdout.log
INFO: Collecting tf logs: /var/log/contrail/control-control
INFO: Collecting tf logs: /var/log/contrail/control-dns
INFO: Collecting tf logs: /var/log/contrail/control-named
INFO: Collecting tf logs: /var/log/contrail/control-nodemgr
INFO: Collecting tf logs: /var/log/contrail/control-provisioner
INFO: Collecting tf logs: /var/log/contrail/database
INFO: Collecting tf logs: /var/log/contrail/database-nodemgr
INFO: Collecting tf logs: /var/log/contrail/database-provisioner
INFO: Collecting tf logs: /var/log/contrail/database-query-engine
INFO: Collecting tf logs: /var/log/contrail/device-manager-dnsmasq
INFO: Collecting tf logs: /var/log/contrail/kafka
INFO: Collecting tf logs: /var/log/contrail/kernel-init
INFO: Collecting tf logs: /var/log/contrail/kubernetes-kube-manager
INFO: Collecting tf logs: /var/log/contrail/node-init
INFO: Collecting tf logs: /var/log/contrail/rabbitmq
INFO: Collecting tf logs: /var/log/contrail/rsyslogd
INFO: Collecting tf logs: /var/log/contrail/vrouter-agent
INFO: Collecting tf logs: /var/log/contrail/vrouter-nodemgr
INFO: Collecting tf logs: /var/log/contrail/vrouter-provisioner
INFO: Collecting tf logs: /var/log/contrail/webui-job
INFO: Collecting tf logs: /var/log/contrail/webui-web
INFO: Collecting tf logs: /var/log/contrail/zookeeper
INFO: Collecting tf logs: save_introspect_info
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8100/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8101/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8102/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8103/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8104/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8108/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8112/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8113/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8114/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8083/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8084/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8085/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8087/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8088/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8096/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8089/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8090/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8091/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:8092/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:5995/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:5920/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting tf logs: introspection request: curl -s http://cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal:5921/Snh_SandeshUVECacheReq?x=NodeStatus
INFO: Collecting core dumps
INFO: content of /var/crash
/var/crash:
total 4
drwxr-xr-x. 2 root root 6 May 9 2023 .
drwxr-xr-x. 22 root root 4096 Aug 26 12:51 ..
INFO: content of /var/crashes
/var/crashes:
total 4
drwxrwxrwx. 2 root root 6 Aug 26 12:51 .
drwxr-xr-x. 22 root root 4096 Aug 26 12:51 ..
INFO: Collecting statuses from cassandra, zookeeper and rabbitmq services
nodetool: Failed to connect to '127.0.0.1:7201' - SecurityException: 'Authentication failed! Credentials required'.
nodetool: Failed to connect to '127.0.0.1:7201' - SecurityException: 'Authentication failed! Credentials required'.
13:28:10.452 [error] ** Node :"contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1" not responding **
** Removing (timedout) connection **
Error: unable to perform an operation on node 'contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1'. Please see diagnostics information and suggestions below.
Most common reasons for this are:
* Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
* CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
* Target node is not running
In addition to the diagnostics info below:
* See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more
* Consult server logs on node contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1
* If target node is configured to use long node names, don't forget to use --longnames with CLI tools
DIAGNOSTICS
===========
attempted to contact: ['contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1']
contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1:
* connected to epmd (port 4369) on cn-jenkins-deploy-platform-ansible-k8s-252-1
* node contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1 up, 'rabbit' application running
Current node details:
* node name: 'rabbitmqcli-1837-contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1'
* effective user's home directory: /var/lib/rabbitmq
* Erlang cookie hash: esZsDxSN6VGbi9JkMSxNZA==
13:34:05.724 [error] ** Node :"contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1" not responding **
** Removing (timedout) connection **
13:39:57.627 [error] ** Node :"contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1" not responding **
** Removing (timedout) connection **
13:45:52.636 [error] ** Node :"contrail@cn-jenkins-deploy-platform-ansible-k8s-252-1" not responding **
** Removing (timedout) connection **
INFO: Collecting kubernetes logs
INFO: Collecting kubernetes object info
INFO: Processing namespace contrail
INFO: collect info for pod
INFO: collect info for daemonset.apps
INFO: collect info for deployment.apps
INFO: collect info for replicaset.apps
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/kube-root-ca.crt
INFO: collect info for endpoints
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: processing secret/contrail-kubemanager-token
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/contrail-kubemanager
INFO: processing serviceaccount/default
INFO: collect info for services
INFO: collect info for jobs
INFO: Processing namespace default
INFO: collect info for pod
INFO: collect info for daemonset.apps
INFO: collect info for deployment.apps
INFO: collect info for replicaset.apps
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/kube-root-ca.crt
INFO: collect info for endpoints
INFO: processing endpoints/ctest-nginx-svc-74006719
INFO: processing endpoints/kubernetes
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/default
INFO: collect info for services
INFO: processing service/ctest-nginx-svc-74006719
INFO: processing service/kubernetes
INFO: collect info for jobs
INFO: Processing namespace kube-node-lease
INFO: collect info for pod
INFO: collect info for daemonset.apps
INFO: collect info for deployment.apps
INFO: collect info for replicaset.apps
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/kube-root-ca.crt
INFO: collect info for endpoints
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/default
INFO: collect info for services
INFO: collect info for jobs
INFO: Processing namespace kube-public
INFO: collect info for pod
INFO: collect info for daemonset.apps
INFO: collect info for deployment.apps
INFO: collect info for replicaset.apps
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/cluster-info
INFO: processing configmap/kube-root-ca.crt
INFO: collect info for endpoints
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/default
INFO: collect info for services
INFO: collect info for jobs
INFO: Processing namespace kube-system
INFO: collect info for pod
INFO: processing pod/coredns-57575c5f89-f28jf
INFO: processing pod/coredns-57575c5f89-kbs7b
INFO: processing pod/dashboard-metrics-scraper-77d78b7997-pp8g8
INFO: processing pod/etcd-cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal
INFO: processing pod/kube-apiserver-cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal
INFO: processing pod/kube-controller-manager-cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal
INFO: processing pod/kube-proxy-8kp5w
INFO: processing pod/kube-scheduler-cn-jenkins-deploy-platform-ansible-k8s-252-1.novalocal
INFO: processing pod/kubernetes-dashboard-779747454c-rmvmz
INFO: collect info for daemonset.apps
INFO: processing daemonset.apps/kube-proxy
INFO: collect info for deployment.apps
INFO: processing deployment.apps/coredns
INFO: processing deployment.apps/dashboard-metrics-scraper
INFO: processing deployment.apps/kubernetes-dashboard
INFO: collect info for replicaset.apps
INFO: processing replicaset.apps/coredns-57575c5f89
INFO: processing replicaset.apps/dashboard-metrics-scraper-77d78b7997
INFO: processing replicaset.apps/kubernetes-dashboard-779747454c
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/coredns
INFO: processing configmap/extension-apiserver-authentication
INFO: processing configmap/kube-proxy
INFO: processing configmap/kube-root-ca.crt
INFO: processing configmap/kubeadm-config
INFO: processing configmap/kubelet-config
INFO: processing configmap/kubernetes-dashboard-settings
INFO: collect info for endpoints
INFO: processing endpoints/dashboard-metrics-scraper
INFO: processing endpoints/kube-dns
INFO: processing endpoints/kubernetes-dashboard
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: processing secret/bootstrap-token-mvak27
INFO: processing secret/kubernetes-dashboard-certs
INFO: processing secret/kubernetes-dashboard-csrf
INFO: processing secret/kubernetes-dashboard-key-holder
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/attachdetach-controller
INFO: processing serviceaccount/bootstrap-signer
INFO: processing serviceaccount/certificate-controller
INFO: processing serviceaccount/clusterrole-aggregation-controller
INFO: processing serviceaccount/coredns
INFO: processing serviceaccount/cronjob-controller
INFO: processing serviceaccount/daemon-set-controller
INFO: processing serviceaccount/default
INFO: processing serviceaccount/deployment-controller
INFO: processing serviceaccount/disruption-controller
INFO: processing serviceaccount/endpoint-controller
INFO: processing serviceaccount/endpointslice-controller
INFO: processing serviceaccount/endpointslicemirroring-controller
INFO: processing serviceaccount/ephemeral-volume-controller
INFO: processing serviceaccount/expand-controller
INFO: processing serviceaccount/generic-garbage-collector
INFO: processing serviceaccount/horizontal-pod-autoscaler
INFO: processing serviceaccount/job-controller
INFO: processing serviceaccount/kube-proxy
INFO: processing serviceaccount/kubernetes-dashboard
INFO: processing serviceaccount/namespace-controller
INFO: processing serviceaccount/node-controller
INFO: processing serviceaccount/persistent-volume-binder
INFO: processing serviceaccount/pod-garbage-collector
INFO: processing serviceaccount/pv-protection-controller
INFO: processing serviceaccount/pvc-protection-controller
INFO: processing serviceaccount/replicaset-controller
INFO: processing serviceaccount/replication-controller
INFO: processing serviceaccount/resourcequota-controller
INFO: processing serviceaccount/root-ca-cert-publisher
INFO: processing serviceaccount/service-account-controller
INFO: processing serviceaccount/service-controller
INFO: processing serviceaccount/statefulset-controller
INFO: processing serviceaccount/token-cleaner
INFO: processing serviceaccount/ttl-after-finished-controller
INFO: processing serviceaccount/ttl-controller
INFO: collect info for services
INFO: processing service/dashboard-metrics-scraper
INFO: processing service/kube-dns
INFO: processing service/kubernetes-dashboard
INFO: collect info for jobs
INFO: Processing namespace kubernetes-dashboard
INFO: collect info for pod
INFO: collect info for daemonset.apps
INFO: collect info for deployment.apps
INFO: collect info for replicaset.apps
INFO: collect info for statefulset.apps
INFO: collect info for configmaps
INFO: processing configmap/kube-root-ca.crt
INFO: collect info for endpoints
INFO: collect info for persistentvolumeclaims
INFO: collect info for secrets
INFO: collect info for serviceaccounts
INFO: processing serviceaccount/default
INFO: collect info for services
INFO: collect info for jobs
INFO: collect info for persistentvolumes
INFO: collect info for customresourcedefinitions
INFO: processing customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io
INFO: collect info for storageclasses
INFO: info collected
/tmp/ansible-logs/logs /tmp/ansible-logs
/tmp/ansible-logs
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
~/.tf/logs/10.0.0.31 ~
~
INFO: Stage logs was run successfully Mon Aug 26 13:46:31 UTC 2024
[update tf stack configuration]
Warning: Permanently added '10.0.0.31' (ED25519) to the list of known hosts.
tf setup profile /home/rocky/.tf/stack.env
DEPLOYER=ansible
CONTRAIL_CONTAINER_TAG=master-8_8_1-4
CONTRAIL_DEPLOYER_CONTAINER_TAG=master-8_8_1-4
CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
DEPLOYER_CONTAINER_REGISTRY=nexus.gz1.opensdn.io:5101
ORCHESTRATOR=kubernetes
OPENSTACK_VERSION="yoga"
CONTROLLER_NODES="10.0.0.31 "
AGENT_NODES="10.0.0.31"
CONTROL_NODES="10.0.0.31 "
SSL_ENABLE="false"
LEGACY_ANALYTICS_ENABLE="true"
HUGE_PAGES_1G=
CONTAINER_RUNTIME=docker
K8S_CA=
DEPLOY_IPA_SERVER=
IPA_PASSWORD=
AUTH_PASSWORD='contrail123'
AUTH_URL=''
INFO: Successful deployment Mon Aug 26 13:46:32 UTC 2024
DEBUG: kill running child jobs:
INFO: Deploy logs finished
INFO: Copy logs from host to workspace
Warning: Permanently added '10.0.0.31' (ECDSA) to the list of known hosts.
~/workspace/collect-logs-and-cleanup@3 ~/workspace/collect-logs-and-cleanup@3
Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts.
Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts.
INFO: Logs collected at http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/81/881/4/templates_27/ansible-k8s
~/workspace/collect-logs-and-cleanup@3
+ /home/jenkins/workspace/collect-logs-and-cleanup@3/src/opensdn-io/tf-jenkins/jobs/devstack/ansible/remove_workers.sh
nova CLI is deprecated and will be removed in a future release
| locked | False |
INFO: do down nodes
+-----------------------------+----------------------------------------------------------+
| Field | Value |
+-----------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | GZ1 |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2024-08-26T12:21:12.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | management=10.0.0.31 |
| config_drive | |
| created | 2024-08-26T12:18:50Z |
| flavor | STD3-4-16 (b6760da7-d124-45c7-a0a7-64f40acae087) |
| hostId | 11d4cd31906e2d6fcd4a7c1036f00d8b3e591f58e1bd24fced14b8aa |
| id | 0101ee00-2606-40a8-9c60-3363fe6e9b68 |
| image | N/A (booted from volume) |
| key_name | worker |
| name | cn_jenkins-deploy-platform-ansible-k8s-252_1 |
| progress | 0 |
| project_id | **** |
| properties | |
| status | ACTIVE |
| updated | 2024-08-26T12:21:17Z |
| user_id | 25d930adb802408885c71e341481886e |
| volumes_attached | id='9244231e-ee2b-4bdd-ad20-880d5c5f7083' |
+-----------------------------+----------------------------------------------------------+
Archiving artifacts
Finished: SUCCESS