Triggered by Gerrit: https://gerrit.opensdn.io/c/opensdn-io/tf-container-builder/+/3087 Obtained pipelines/main.groovy from git https://github.com/opensdn-io/tf-jenkins.git [Pipeline] Start of Pipeline [Pipeline] timestamps [Pipeline] { [Pipeline] timeout [2026-09-29T15:42:47.201Z] Timeout set to expire in 8 hr 0 min [Pipeline] { [Pipeline] node [2026-09-29T15:42:47.258Z] Running on slave-openstack-gz1_1 in /home/jenkins/workspace/pipeline-post-merge [Pipeline] { [Pipeline] stage [Pipeline] { (init) [Pipeline] cleanWs [2026-09-29T15:42:47.349Z] [WS-CLEANUP] Deleting project workspace... [2026-09-29T15:42:47.349Z] [WS-CLEANUP] Deferred wipeout is disabled by the job configuration... [2026-09-29T15:42:47.454Z] [WS-CLEANUP] done [Pipeline] checkout [2026-09-29T15:42:47.486Z] The recommended git tool is: git [2026-09-29T15:42:47.491Z] No credentials specified [2026-09-29T15:42:47.494Z] Cloning the remote Git repository [2026-09-29T15:42:47.497Z] Cloning repository https://github.com/opensdn-io/tf-jenkins.git [2026-09-29T15:42:47.497Z] > git init /home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins # timeout=10 [2026-09-29T15:42:47.502Z] Fetching upstream changes from https://github.com/opensdn-io/tf-jenkins.git [2026-09-29T15:42:47.502Z] > git --version # timeout=10 [2026-09-29T15:42:47.505Z] > git --version # 'git version 2.25.1' [2026-09-29T15:42:47.505Z] > git fetch --tags --force --progress -- https://github.com/opensdn-io/tf-jenkins.git +refs/heads/*:refs/remotes/origin/* # timeout=10 [2026-09-29T15:42:48.973Z] Avoid second fetch [2026-09-29T15:42:48.979Z] Checking out Revision 90c59032cae9af68279c9c2f84375057e81bdd5c (refs/remotes/origin/master) [2026-09-29T15:42:49.046Z] Commit message: "Merge "reuse boot volume if instance create fails"" [2026-09-29T15:42:48.962Z] > git config remote.origin.url https://github.com/opensdn-io/tf-jenkins.git # timeout=10 [2026-09-29T15:42:48.967Z] > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 [2026-09-29T15:42:48.974Z] > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 [2026-09-29T15:42:48.980Z] > git config core.sparsecheckout # timeout=10 [2026-09-29T15:42:48.983Z] > git checkout -f 90c59032cae9af68279c9c2f84375057e81bdd5c # timeout=10 [2026-09-29T15:42:49.048Z] > git rev-list --no-walk 90c59032cae9af68279c9c2f84375057e81bdd5c # timeout=10 [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/utils/gerrit.groovy) [Pipeline] } [Pipeline] // load [Pipeline] echo [2026-09-29T15:42:49.603Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-09-29T15:42:49.929Z] + /home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/infra/gerrit/resolve_patchsets.py --gerrit https://gerrit.opensdn.io/ --review I444656a7a69f1dcce7dc52988c866e431f5e684f --branch master --changed_files --output /home/jenkins/workspace/pipeline-post-merge/patchsets-info.json [Pipeline] archiveArtifacts [2026-09-29T15:42:49.962Z] Archiving artifacts [Pipeline] sh [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/utils/gerrit.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/constants.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/utils/config.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/utils/jobs.groovy) [Pipeline] } [Pipeline] // load [Pipeline] load [Pipeline] { (/home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/pipelines/utils/gate.groovy) [Pipeline] } [Pipeline] // load [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Pre-build) [Pipeline] echo [2026-09-29T15:42:51.730Z] Logs URL: http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/87/3087/1/post-merge_969 [Pipeline] echo [2026-09-29T15:42:51.757Z] terminate_runs_by_review_number: start [Pipeline] echo [2026-09-29T15:42:54.322Z] terminate_runs_by_review_number: terminated builds = [] [Pipeline] sh [Pipeline] echo [2026-09-29T15:42:54.651Z] Pipeline to run: post-merge [Pipeline] sh [Pipeline] sh [Pipeline] archiveArtifacts [2026-09-29T15:42:55.270Z] Archiving artifacts [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] readYaml [Pipeline] echo [2026-09-29T15:42:56.153Z] Streams from config: [:] [Pipeline] echo [2026-09-29T15:42:56.171Z] Jobs from config: [update-caches:[:]] [Pipeline] echo [2026-09-29T15:42:56.186Z] Post Jobs from config: [:] [Pipeline] echo [2026-09-29T15:42:56.223Z] Notify gerrit verified=null, submit=false, msg= [2026-09-29T15:42:56.223Z] TF CI Build Started (post-merge) https://jenkins.opensdn.io/job/pipeline-post-merge/969/ [Pipeline] withCredentials [2026-09-29T15:42:56.252Z] Masking supported pattern matches of $GERRIT_API_PASSWORD [Pipeline] { [Pipeline] echo [2026-09-29T15:42:56.305Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-09-29T15:42:56.611Z] + /home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/infra/gerrit/notify.py --gerrit https://gerrit.opensdn.io/ --user tf-gerrit --password **** --review I444656a7a69f1dcce7dc52988c866e431f5e684f --patchset 1 --branch master --message 'TF CI Build Started (post-merge) https://jenkins.opensdn.io/job/pipeline-post-merge/969/' [2026-09-29T15:42:56.868Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-container-builder~master~I444656a7a69f1dcce7dc52988c866e431f5e684f', 'project': 'opensdn-io/tf-container-builder', 'branch': 'master', 'attention_set': {}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-09-29 12:41:54.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000018': {'account': {'_account_id': 1000018}, 'last_update': '2026-09-29 15:42:29.000000000', 'reason': 'Change was submitted'}, '1000012': {'account': {'_account_id': 1000012}, 'last_update': '2026-09-29 15:42:29.000000000', 'reason': 'Change was submitted'}}, 'hashtags': [], 'change_id': 'I444656a7a69f1dcce7dc52988c866e431f5e684f', 'subject': 'Persist underlay addressing of the vhost0 physical interface', 'status': 'MERGED', 'created': '2026-09-17 09:31:25.000000000', 'updated': '2026-09-29 15:42:29.000000000', 'submitted': '2026-09-29 15:42:29.000000000', 'submitter': {'_account_id': 1000001}, 'insertions': 290, 'deletions': 20, 'total_comment_count': 1, 'unresolved_comment_count': 0, 'has_review_started': True, 'submission_id': '3087', 'meta_rev_id': 'b92a8bf21dd001e78691555d10b6343672d29ed5', '_number': 3087, 'owner': {'_account_id': 1000018}, 'labels': {'Code-Review': {'all': [{'value': 2, 'date': '2026-09-29 12:41:54.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000003}, {'value': 0, '_account_id': 1000001}, {'value': 1, 'date': '2026-09-21 06:33:56.000000000', 'permitted_voting_range': {'min': 1, 'max': 1}, '_account_id': 1000009}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, '_account_id': 1000003}, {'value': 2, 'date': '2026-09-29 15:42:16.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000001}, {'value': 0, '_account_id': 1000009}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'permitted_labels': {'Code-Review': [' 0', '+1', '+2'], 'Verified': ['+2']}, 'removable_labels': {}, 'removable_reviewers': [{'_account_id': 1000012}], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}, {'_account_id': 1000009}, {'_account_id': 1000012}]}, 'pending_reviewers': {}, 'current_revision': '9a92cf24515d979a9760064210988369d53f1be0', 'revisions': {'9a92cf24515d979a9760064210988369d53f1be0': {'kind': 'REWORK', '_number': 1, 'created': '2026-09-17 09:31:25.000000000', 'uploader': {'_account_id': 1000018}, 'ref': 'refs/changes/87/3087/1', 'fetch': {'ssh': {'url': 'ssh://tf-gerrit@gerrit.opensdn.io:29418/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}, 'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}, 'http': {'url': 'https://tf-gerrit@gerrit.opensdn.io/a/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}}, 'commit': {'parents': [{'commit': '96a28482c89b420f5f5d7d9fc01f5b1983e86426', 'subject': 'Merge "upgrade zookeeper to 3.9,5"'}], 'author': {'name': 'Gleb Zimin', 'email': 'gzimin@mirantis.com', 'date': '2026-09-08 13:00:53.000000000', 'tz': 120}, 'committer': {'name': 'Gleb Zimin', 'email': 'gzimin@mirantis.com', 'date': '2026-09-17 09:30:10.000000000', 'tz': 120}, 'subject': 'Persist underlay addressing of the vhost0 physical interface', 'message': 'Persist underlay addressing of the vhost0 physical interface\n\ninit_vhost0 moves the addresses and routes of the physical interface\nto vhost0 and remove_vhost0 moves them back, so in kernel mode the\nonly copy of that addressing is the one held in the kernel. When the\nagent goes away without the move back - the node is disabled and\nre-enabled, rebooted, or the container is killed - the copy is lost\ntogether with vhost0 and the host network config is not re-applied,\nso the address ends up on no interface at all.\n\nEvery following start then reads no address from the physical\ninterface, creates a vhost0 without a CIDR and waits for it for 60\ncycles until the liveness probe kills the container, which removes\nvhost0 and restores nothing. The node stays down and cannot recover\non its own.\n\nKeep a copy of the addressing in /var/lib/contrail (/var/run is not\nmounted into the agent container in kernel mode and does not survive\na reboot on the host), and use it wherever the live state is gone:\n\n- refresh it while vhost0 still holds the addressing, and on every\n start that finds vhost0 already up, so that an address changed in\n the meantime is picked up\n- fall back to it when the physical interface has no address, and\n fail with an explicit error when there is nothing to fall back to\n instead of bringing up an unusable vhost0\n- fall back to it in remove_vhost0 as well, when vhost0 has no\n address of its own to move back, skipping the addresses the host\n network config has already put back on the physical interface\n- remove a leftover vhost0 of a previous ungraceful death before\n creating it again, so that remove_vhost0 gets the chance to put\n the addressing back on the physical interface first. Only a\n leftover without any addressing is removed: check_vhost0 looks for\n an IPv4 address and so fails for a working IPv6-only vhost0 as\n well, and on the ifup path an address that does not come from the\n ifcfg files could not be brought back\n\nOnly the path that initializes vhost0 manually is affected. The\nifcfg one takes the addressing from the ifcfg files and expects an\nunconfigured physical interface, so the condition that picks between\nthe two is now a function that can be evaluated before the\naddressing is read, and the leftover cleanup is left to ifup there.\n\nAlong the way:\n\n- skip IPv6 link-local addresses when moving addresses between the\n physical interface and vhost0. vhost0 inherits the MAC, so the\n kernel has already assigned the same address to it and the move\n can only fail with "ipv6: address already assigned", which also\n hid the missing global address in the logs\n- do not take an IPv4 link-local address for a configured one when\n deciding whether the physical interface has an address at all, it\n means DHCP has failed\n- delete an address from the interface that actually holds it, which\n is neither the resolved one when it has been restored from the\n saved copy, nor the same one across the removal of vhost0: the\n physical interface is resolved by the MAC of vhost0 while it\n exists and by the configuration once it is gone\n- report a single address that cannot be moved as a warning and fail\n the initialization only when vhost0 has ended up with no address\n at all. Failures were silently lost with the subshell of the\n pipeline before, and making every one of them fatal would turn a\n duplicate address into a permanent crash loop\n- skip empty lines in gen_ip_addr_add_cmd, it generated a bare\n "ip address add dev " for an interface without addresses\n- keep only the routes nothing recreates on its own. The ones vrouter\n injects into vhost0 (proto 109) are put back there on every start,\n and the teardown copied them to the physical interface, where the\n next start read them back as underlay ones and kept spreading them.\n The connected route of the subnet (proto kernel) comes with the\n address, and saving it would pin the source address it was\n generated with. What is left is what only the host network config\n can bring back, such as the static route to the underlay subnet of\n another rack\n- say in the log where the addressing comes from and when the saved\n copy is written, read or missing. A start that needs no recovery\n looked exactly like one that silently had nothing to recover\n\nCo-Authored-By: Claude Opus 5 (1M context) \nChange-Id: I444656a7a69f1dcce7dc52988c866e431f5e684f\n'}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'CLOSED', 'labels': [{'label': 'Code-Review', 'status': 'OK', 'applied_by': {'_account_id': 1000003}}, {'label': 'Verified', 'status': 'OK', 'applied_by': {'_account_id': 1000001}}]}]} [2026-09-29T15:42:56.868Z] DEBUG: push message data: {'labels': {}, 'message': 'TF CI Build Started (post-merge) https://jenkins.opensdn.io/job/pipeline-post-merge/969/'} [Pipeline] } [Pipeline] // withCredentials [Pipeline] echo [2026-09-29T15:42:56.947Z] Notify discord for new job is started [Pipeline] echo [2026-09-29T15:42:56.988Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] echo [2026-09-29T15:42:57.012Z] Notify discord msg= [2026-09-29T15:42:57.012Z] The review was merged. [2026-09-29T15:42:57.012Z] [Persist underlay addressing of the vhost0 physical](https://gerrit.opensdn.io/c/opensdn-io/tf-container-builder/+/3087) [Pipeline] withCredentials [2026-09-29T15:42:57.037Z] Masking supported pattern matches of $DISCORD_WEBHOOK_URL [Pipeline] { [Pipeline] writeFile [Pipeline] sh [2026-09-29T15:42:57.448Z] % Total % Received % Xferd Average Speed Time Time Time Current [2026-09-29T15:42:57.448Z] Dload Upload Total Spent Left Speed [2026-09-29T15:42:57.708Z] [2026-09-29T15:42:57.708Z] 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 [2026-09-29T15:42:57.708Z] 100 191 0 0 100 191 0 530 --:--:-- --:--:-- --:--:-- 530 [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] parallel [Pipeline] { (Branch: job-update-caches) [Pipeline] stage [Pipeline] { (update-caches) [Pipeline] echo [2026-09-29T15:42:57.976Z] JOB update-caches: entering run_job [Pipeline] echo [2026-09-29T15:42:58.007Z] JOB update-caches: vars empty. do not store vars file. [Pipeline] echo [2026-09-29T15:42:58.038Z] JOB update-caches: Starting job: update-caches rnd: #22114 [Pipeline] timeout [2026-09-29T15:42:58.055Z] Timeout set to expire in 3 hr 0 min [Pipeline] { [Pipeline] build [2026-09-29T15:42:58.109Z] Scheduling project: update-caches [2026-09-29T15:43:06.379Z] Starting building: update-caches #501 [2026-09-29T16:46:44.392Z] Build update-caches #501 completed: SUCCESS [Pipeline] echo [2026-09-29T16:46:44.414Z] JOB update-caches: Finished with SUCCESS [Pipeline] } [Pipeline] // timeout [Pipeline] copyArtifacts [2026-09-29T16:46:44.500Z] Copied 0 artifacts from "update-caches" build number 501 [Pipeline] echo [2026-09-29T16:46:44.530Z] JOB update-caches: Collected artifacts: [Pipeline] sh [2026-09-29T16:46:44.822Z] + ls -la update-caches-22114 [2026-09-29T16:46:44.822Z] total 8 [2026-09-29T16:46:44.822Z] drwxr-xr-x 2 jenkins docker 4096 Sep 29 16:46 . [2026-09-29T16:46:44.822Z] drwxr-xr-x 4 jenkins docker 4096 Sep 29 16:46 .. [Pipeline] withCredentials [2026-09-29T16:46:44.866Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh [2026-09-29T16:46:45.173Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-09-29T16:46:46.542Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [Pipeline] sh [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // parallel [Pipeline] echo [2026-09-29T16:46:47.630Z] Jobs results: [update-caches:[job-rnd:22114, number:501, started:1790696586378, duration:3817922, result:SUCCESS]] [Pipeline] stage [Pipeline] { (gerrit vote) [Pipeline] echo [2026-09-29T16:46:47.697Z] Evaluated results for update-caches = [results:[SUCCESS], duration:3817922] [Pipeline] echo [2026-09-29T16:46:47.733Z] Notify gerrit verified=null, submit=false, msg= [2026-09-29T16:46:47.733Z] TF CI Build Succeeded (post-merge) in 1h 3m 56s [2026-09-29T16:46:47.733Z] [2026-09-29T16:46:47.733Z] Succeeded checks: [2026-09-29T16:46:47.733Z] - update-caches http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/87/3087/1/post-merge_969/update-caches : SUCCESS in 1h 3m 37s [Pipeline] withCredentials [2026-09-29T16:46:47.759Z] Masking supported pattern matches of $GERRIT_API_PASSWORD [Pipeline] { [Pipeline] echo [2026-09-29T16:46:47.855Z] INFO: resolved gerrit URL is https://gerrit.opensdn.io/ [Pipeline] sh [2026-09-29T16:46:48.152Z] + /home/jenkins/workspace/pipeline-post-merge/src/opensdn-io/tf-jenkins/infra/gerrit/notify.py --gerrit https://gerrit.opensdn.io/ --user tf-gerrit --password **** --review I444656a7a69f1dcce7dc52988c866e431f5e684f --patchset 1 --branch master --message 'TF CI Build Succeeded (post-merge) in 1h 3m 56s [2026-09-29T16:46:48.152Z] [2026-09-29T16:46:48.152Z] Succeeded checks: [2026-09-29T16:46:48.152Z] - update-caches http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/87/3087/1/post-merge_969/update-caches : SUCCESS in 1h 3m 37s' [2026-09-29T16:46:48.153Z] DEBUG: Change: {'id': 'opensdn-io%2Ftf-container-builder~master~I444656a7a69f1dcce7dc52988c866e431f5e684f', 'project': 'opensdn-io/tf-container-builder', 'branch': 'master', 'attention_set': {}, 'removed_from_attention_set': {'1000003': {'account': {'_account_id': 1000003}, 'last_update': '2026-09-29 12:41:54.000000000', 'reason': ' replied on the change', 'reason_account': {'_account_id': 1000003}}, '1000018': {'account': {'_account_id': 1000018}, 'last_update': '2026-09-29 15:42:29.000000000', 'reason': 'Change was submitted'}, '1000012': {'account': {'_account_id': 1000012}, 'last_update': '2026-09-29 15:42:29.000000000', 'reason': 'Change was submitted'}}, 'hashtags': [], 'change_id': 'I444656a7a69f1dcce7dc52988c866e431f5e684f', 'subject': 'Persist underlay addressing of the vhost0 physical interface', 'status': 'MERGED', 'created': '2026-09-17 09:31:25.000000000', 'updated': '2026-09-29 15:42:56.000000000', 'submitted': '2026-09-29 15:42:29.000000000', 'submitter': {'_account_id': 1000001}, 'insertions': 290, 'deletions': 20, 'total_comment_count': 1, 'unresolved_comment_count': 0, 'has_review_started': True, 'submission_id': '3087', 'meta_rev_id': '5ee6df12273368d23a222839dcb727b4d860528a', '_number': 3087, 'owner': {'_account_id': 1000018}, 'labels': {'Code-Review': {'all': [{'value': 2, 'date': '2026-09-29 12:41:54.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000003}, {'value': 0, '_account_id': 1000001}, {'value': 1, 'date': '2026-09-21 06:33:56.000000000', 'permitted_voting_range': {'min': 1, 'max': 1}, '_account_id': 1000009}], 'values': {'-2': 'This shall not be submitted', '-1': 'I would prefer this is not submitted as is', ' 0': 'No score', '+1': 'Looks good to me, but someone else must approve', '+2': 'Looks good to me, approved'}, 'description': '', 'default_value': 0}, 'Verified': {'all': [{'value': 0, '_account_id': 1000003}, {'value': 2, 'date': '2026-09-29 15:42:16.000000000', 'permitted_voting_range': {'min': 2, 'max': 2}, '_account_id': 1000001}, {'value': 0, '_account_id': 1000009}], 'values': {'-2': 'Fails', '-1': 'Fails', ' 0': 'No score', '+1': 'Verified', '+2': 'Gating'}, 'description': '', 'default_value': 0}}, 'permitted_labels': {'Code-Review': [' 0', '+1', '+2'], 'Verified': ['+2']}, 'removable_labels': {}, 'removable_reviewers': [{'_account_id': 1000012}], 'reviewers': {'REVIEWER': [{'_account_id': 1000001}, {'_account_id': 1000003}, {'_account_id': 1000009}, {'_account_id': 1000012}]}, 'pending_reviewers': {}, 'current_revision': '9a92cf24515d979a9760064210988369d53f1be0', 'revisions': {'9a92cf24515d979a9760064210988369d53f1be0': {'kind': 'REWORK', '_number': 1, 'created': '2026-09-17 09:31:25.000000000', 'uploader': {'_account_id': 1000018}, 'ref': 'refs/changes/87/3087/1', 'fetch': {'ssh': {'url': 'ssh://tf-gerrit@gerrit.opensdn.io:29418/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}, 'anonymous http': {'url': 'https://gerrit.opensdn.io/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}, 'http': {'url': 'https://tf-gerrit@gerrit.opensdn.io/a/opensdn-io/tf-container-builder', 'ref': 'refs/changes/87/3087/1'}}, 'commit': {'parents': [{'commit': '96a28482c89b420f5f5d7d9fc01f5b1983e86426', 'subject': 'Merge "upgrade zookeeper to 3.9,5"'}], 'author': {'name': 'Gleb Zimin', 'email': 'gzimin@mirantis.com', 'date': '2026-09-08 13:00:53.000000000', 'tz': 120}, 'committer': {'name': 'Gleb Zimin', 'email': 'gzimin@mirantis.com', 'date': '2026-09-17 09:30:10.000000000', 'tz': 120}, 'subject': 'Persist underlay addressing of the vhost0 physical interface', 'message': 'Persist underlay addressing of the vhost0 physical interface\n\ninit_vhost0 moves the addresses and routes of the physical interface\nto vhost0 and remove_vhost0 moves them back, so in kernel mode the\nonly copy of that addressing is the one held in the kernel. When the\nagent goes away without the move back - the node is disabled and\nre-enabled, rebooted, or the container is killed - the copy is lost\ntogether with vhost0 and the host network config is not re-applied,\nso the address ends up on no interface at all.\n\nEvery following start then reads no address from the physical\ninterface, creates a vhost0 without a CIDR and waits for it for 60\ncycles until the liveness probe kills the container, which removes\nvhost0 and restores nothing. The node stays down and cannot recover\non its own.\n\nKeep a copy of the addressing in /var/lib/contrail (/var/run is not\nmounted into the agent container in kernel mode and does not survive\na reboot on the host), and use it wherever the live state is gone:\n\n- refresh it while vhost0 still holds the addressing, and on every\n start that finds vhost0 already up, so that an address changed in\n the meantime is picked up\n- fall back to it when the physical interface has no address, and\n fail with an explicit error when there is nothing to fall back to\n instead of bringing up an unusable vhost0\n- fall back to it in remove_vhost0 as well, when vhost0 has no\n address of its own to move back, skipping the addresses the host\n network config has already put back on the physical interface\n- remove a leftover vhost0 of a previous ungraceful death before\n creating it again, so that remove_vhost0 gets the chance to put\n the addressing back on the physical interface first. Only a\n leftover without any addressing is removed: check_vhost0 looks for\n an IPv4 address and so fails for a working IPv6-only vhost0 as\n well, and on the ifup path an address that does not come from the\n ifcfg files could not be brought back\n\nOnly the path that initializes vhost0 manually is affected. The\nifcfg one takes the addressing from the ifcfg files and expects an\nunconfigured physical interface, so the condition that picks between\nthe two is now a function that can be evaluated before the\naddressing is read, and the leftover cleanup is left to ifup there.\n\nAlong the way:\n\n- skip IPv6 link-local addresses when moving addresses between the\n physical interface and vhost0. vhost0 inherits the MAC, so the\n kernel has already assigned the same address to it and the move\n can only fail with "ipv6: address already assigned", which also\n hid the missing global address in the logs\n- do not take an IPv4 link-local address for a configured one when\n deciding whether the physical interface has an address at all, it\n means DHCP has failed\n- delete an address from the interface that actually holds it, which\n is neither the resolved one when it has been restored from the\n saved copy, nor the same one across the removal of vhost0: the\n physical interface is resolved by the MAC of vhost0 while it\n exists and by the configuration once it is gone\n- report a single address that cannot be moved as a warning and fail\n the initialization only when vhost0 has ended up with no address\n at all. Failures were silently lost with the subshell of the\n pipeline before, and making every one of them fatal would turn a\n duplicate address into a permanent crash loop\n- skip empty lines in gen_ip_addr_add_cmd, it generated a bare\n "ip address add dev " for an interface without addresses\n- keep only the routes nothing recreates on its own. The ones vrouter\n injects into vhost0 (proto 109) are put back there on every start,\n and the teardown copied them to the physical interface, where the\n next start read them back as underlay ones and kept spreading them.\n The connected route of the subnet (proto kernel) comes with the\n address, and saving it would pin the source address it was\n generated with. What is left is what only the host network config\n can bring back, such as the static route to the underlay subnet of\n another rack\n- say in the log where the addressing comes from and when the saved\n copy is written, read or missing. A start that needs no recovery\n looked exactly like one that silently had nothing to recover\n\nCo-Authored-By: Claude Opus 5 (1M context) \nChange-Id: I444656a7a69f1dcce7dc52988c866e431f5e684f\n'}}}, 'requirements': [], 'submit_records': [{'rule_name': 'gerrit~DefaultSubmitRule', 'status': 'CLOSED', 'labels': [{'label': 'Code-Review', 'status': 'OK', 'applied_by': {'_account_id': 1000003}}, {'label': 'Verified', 'status': 'OK', 'applied_by': {'_account_id': 1000001}}]}]} [2026-09-29T16:46:48.153Z] DEBUG: push message data: {'labels': {}, 'message': 'TF CI Build Succeeded (post-merge) in 1h 3m 56s\n\nSucceeded checks:\n- update-caches http://nexus.gz1.opensdn.io:8082/jenkins_logs/gerrit/87/3087/1/post-merge_969/update-caches : SUCCESS in 1h 3m 37s'} [Pipeline] } [Pipeline] // withCredentials [Pipeline] sh [Pipeline] archiveArtifacts [2026-09-29T16:46:48.540Z] Archiving artifacts [Pipeline] withCredentials [2026-09-29T16:46:48.597Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] writeFile [Pipeline] sh [2026-09-29T16:46:48.940Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [2026-09-29T16:46:49.503Z] Warning: Permanently added 'nexus.gz1.opensdn.io,212.233.90.199' (ECDSA) to the list of known hosts. [Pipeline] } [Pipeline] // withCredentials [Pipeline] } [Pipeline] // stage [Pipeline] echo [2026-09-29T16:46:49.922Z] URL of console output = https://jenkins.opensdn.io/job/pipeline-post-merge/969/consoleText [Pipeline] withCredentials [2026-09-29T16:46:49.957Z] Masking supported pattern matches of $LOGS_HOST_SSH_KEY [Pipeline] { [Pipeline] sh