INFO: =================== Mon Oct 21 01:54:13 UTC 2024 =================== INFO: =================== Mon Oct 21 01:54:13 UTC 2024 =================== INFO: K8S deployment, use K8S facilities for cert-generation INFO: openssl config file [req] default_bits = 2048 prompt = no default_md = sha256 default_days = 375 req_extensions = v3_req distinguished_name = req_distinguished_name x509_extensions = v3_ca [ req_distinguished_name ] commonName = Contrail countryName = US stateOrProvinceName = CA organizationName = JuniperNetworks organizationalUnitName = JuniperCA localityName = Sunnyvale [ v3_req ] basicConstraints = CA:false keyUsage = nonRepudiation, digitalSignature, keyEncipherment subjectAltName = @alt_names [ alt_names ] DNS.1 = cn-jenkins-deploy-platform-k8s-manifests-102-1 DNS.2 = cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local IP.1 = 10.0.0.36 IP.2 = 10.96.0.1 IP.3 = 10.96.0.3 IP.4 = 172.17.0.1 [ ca ] default_ca = CA_default [ CA_default ] # Directory and file locations. dir = /tmp/contrail_ssl_gen crl_dir = $dir/crl new_certs_dir = $dir/certs database = $dir/index.txt serial = $dir/serial.txt RANDFILE = $dir/.rand # For certificate revocation lists. crlnumber = $dir/crlnumber crl = $dir/crl/crl.pem crl_extensions = crl_ext default_crl_days = 30 # The root key and root certificate. private_key = /tmp/contrail_ssl_gen/certs/ca.key.pem certificate = /etc/contrail/webui_ssl/ca-cert.pem # SHA-1 is deprecated, so use SHA-2 instead. default_md = sha256 name_opt = ca_default cert_opt = ca_default default_days = 375 preserve = no policy = policy_optional [ policy_optional ] countryName = optional stateOrProvinceName = optional organizationName = optional organizationalUnitName = optional commonName = supplied emailAddress = optional localityName = optional [ v3_ca] # Extensions for a typical CA # PKIX recommendation. subjectKeyIdentifier=hash authorityKeyIdentifier=keyid:always,issuer:always basicConstraints = CA:true [ crl_ext ] authorityKeyIdentifier=keyid:always,issuer:always Generating RSA private key, 2048 bit long modulus .......................................................................................+++ ....................+++ e is 65537 (0x10001) INFO: base k8s curl url: INFO: K8S: remove old CSR if any % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 631 100 631 0 0 914 0 --:--:-- --:--:-- --:--:-- 914 { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "certificatesigningrequests.certificates.k8s.io \"contrail-node.cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local\" is forbidden: User \"system:serviceaccount:kube-system:default\" cannot delete resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope", "reason": "Forbidden", "details": { "name": "contrail-node.cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local", "group": "certificates.k8s.io", "kind": "certificatesigningrequests" }, "code": 403 }INFO: send new CSR for signing by K8S authority % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 18 2548 100 462 0 0 1358 0 --:--:-- --:--:-- --:--:-- 1362 { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "certificatesigningrequests.certificates.k8s.io is forbidden: User \"system:serviceaccount:kube-system:default\" cannot create resource \"certificatesigningrequests\" in API group \"certificates.k8s.io\" at the cluster scope", "reason": "Forbidden", "details": { "group": "certificates.k8s.io", "kind": "certificatesigningrequests" }, "code": 403 }INFO: approve CSR % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 21 2927 100 640 0 0 2074 0 --:--:-- --:--:-- --:--:-- 2071 { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "certificatesigningrequests.certificates.k8s.io \"contrail-node.cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local\" is forbidden: User \"system:serviceaccount:kube-system:default\" cannot update resource \"certificatesigningrequests/approval\" in API group \"certificates.k8s.io\" at the cluster scope", "reason": "Forbidden", "details": { "name": "contrail-node.cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local", "group": "certificates.k8s.io", "kind": "certificatesigningrequests" }, "code": 403 }INFO: download approved certificate % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 628 100 628 0 0 2756 0 --:--:-- --:--:-- --:--:-- 2766 WARNING: failed to sign CSR by K8S CA. The Kubernetes controller responsible of approving the certificates could be disabled. Fallback to local self-signed CA. Generating RSA private key, 4096 bit long modulus ...................................................................................................................................................................................................................................................................................................................................................................++ ............................................................................................................................................................................................................................................................................................................................++ e is 65537 (0x10001) Using configuration from /tmp/contrail_ssl_gen/contrail_openssl.cfg Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Oct 21 01:54:22 2024 GMT Not After : Oct 21 01:54:22 2025 GMT Subject: countryName = US stateOrProvinceName = CA organizationName = JuniperNetworks organizationalUnitName = JuniperCA commonName = Contrail localityName = Sunnyvale X509v3 extensions: X509v3 Basic Constraints: CA:FALSE X509v3 Key Usage: Digital Signature, Non Repudiation, Key Encipherment X509v3 Subject Alternative Name: DNS:cn-jenkins-deploy-platform-k8s-manifests-102-1, DNS:cn-jenkins-deploy-platform-k8s-manifests-102-1.cluster.local, IP Address:10.0.0.36, IP Address:10.96.0.1, IP Address:10.96.0.3, IP Address:172.17.0.1 Certificate is to be certified until Oct 21 01:54:22 2025 GMT (365 days) Sign the certificate? [y/n]: 1 out of 1 certificate requests certified, commit? [y/n]Write out database with 1 new entries Data Base Updated INFO: config /etc/contrail/config.global.js /* * Copyright (c) 2014 Juniper Networks, Inc. All rights reserved. */ var config = {}; config.orchestration = {}; config.orchestration.Manager = "none"; config.orchestrationModuleEndPointFromConfig = false; config.contrailEndPointFromConfig = true; config.regionsFromConfig = false; config.endpoints = {}; config.endpoints.apiServiceType = "ApiServer"; config.endpoints.opServiceType = "OpServer"; config.regions = {}; config.regions.RegionOne = "http://127.0.0.1:5000/v2.0"; config.serviceEndPointTakePublicURL = true; config.networkManager = {}; config.networkManager.ip = "127.0.0.1"; config.networkManager.port = "9696"; config.networkManager.authProtocol = "http"; config.networkManager.apiVersion = []; config.networkManager.strictSSL = false; config.networkManager.ca = ""; config.imageManager = {}; config.imageManager.ip = "127.0.0.1"; config.imageManager.port = "9292"; config.imageManager.authProtocol = "http"; config.imageManager.apiVersion = ['v1', 'v2']; config.imageManager.strictSSL = false; config.imageManager.ca = ""; config.computeManager = {}; config.computeManager.ip = "127.0.0.1"; config.computeManager.port = "8774"; config.computeManager.authProtocol = "http"; config.computeManager.apiVersion = ['v1.1', 'v2']; config.computeManager.strictSSL = false; config.computeManager.ca = ""; config.identityManager = {}; config.identityManager.ip = "127.0.0.1"; config.identityManager.port = "5000"; config.identityManager.authProtocol = "http"; config.identityManager.apiVersion = ['v3']; config.identityManager.strictSSL = false; config.identityManager.ca = ""; config.storageManager = {}; config.storageManager.ip = "127.0.0.1"; config.storageManager.port = "8776"; config.storageManager.authProtocol = "http"; config.storageManager.apiVersion = ['v1']; config.storageManager.strictSSL = false; config.storageManager.ca = ""; config.cnfg = {}; config.cnfg.server_ip = ['10.0.0.36']; config.cnfg.server_port = "8082"; config.cnfg.authProtocol = "http"; config.cnfg.strictSSL = false; config.cnfg.ca = "/etc/contrail/ssl/certs/ca-cert.pem"; config.cnfg.statusURL = '/global-system-configs'; config.analytics = {}; config.analytics.server_ip = ['10.0.0.36']; config.analytics.server_port = "8081"; config.analytics.authProtocol = "http"; config.analytics.strictSSL = false; config.analytics.ca = "/etc/contrail/ssl/certs/ca-cert.pem"; config.analytics.statusURL = '/analytics/uves/bgp-peers'; config.dns = {}; config.dns.server_ip = ['10.0.0.36']; config.dns.server_port = '8092'; config.dns.statusURL = '/Snh_PageReq?x=AllEntries%20VdnsServersReq'; config.introspect = {}; config.introspect.ssl = {}; config.introspect.ssl.enabled = false; config.introspect.ssl.key = '/etc/contrail/ssl/private/server-privkey.pem'; config.introspect.ssl.cert = '/etc/contrail/ssl/certs/server.pem'; config.introspect.ssl.ca = '/etc/contrail/ssl/certs/ca-cert.pem'; config.introspect.ssl.strictSSL = false; config.jobServer = {}; config.jobServer.server_ip = '127.0.0.1'; config.jobServer.server_port = '3000'; config.files = {}; config.files.download_path = '/tmp'; config.cassandra = {}; config.cassandra.server_ips = ['10.0.0.36']; config.cassandra.server_port = '9041'; config.cassandra.enable_edit = false; config.cassandra.use_ssl = false; config.cassandra.ca_certs = '/etc/contrail/ssl/certs/ca-cert.pem'; config.kue = {}; config.kue.ui_port = '3002' config.webui_addresses = ['10.0.0.36']; config.insecure_access = false; config.http_port = '8180'; config.https_port = '8143'; config.require_auth = false; config.node_worker_count = 1; config.maxActiveJobs = 10; config.redisDBIndex = 3; config.CONTRAIL_SERVICE_RETRY_TIME = 300000; //5 minutes config.redis_server_port = '6379'; config.redis_server_ip = '127.0.0.1'; config.redis_dump_file = '/var/lib/redis/dump-webui.rdb'; config.redis_password = ''; config.logo_file = '/opt/contrail/images/logo.png'; config.favicon_file = '/opt/contrail/images/favicon.ico'; config.featurePkg = {}; config.featurePkg.webController = {}; config.featurePkg.webController.path = '/usr/src/contrail/contrail-web-controller'; config.featurePkg.webController.enable = true; config.qe = {}; config.qe.enable_stat_queries = false; config.logs = {}; config.logs.level = 'debug'; config.getDomainProjectsFromApiServer = false; config.network = {}; config.network.L2_enable = false; config.getDomainsFromApiServer = false; config.jsonSchemaPath = "/usr/src/contrail/contrail-web-core/src/serverroot/configJsonSchemas"; config.server_options = {}; config.server_options.key_file = '/etc/contrail/webui_ssl/cs-key.pem'; config.server_options.cert_file = '/etc/contrail/webui_ssl/cs-cert.pem'; config.server_options.ciphers = 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:AES256-SHA'; module.exports = config; config.staticAuth = []; config.staticAuth[0] = {}; config.staticAuth[0].username = 'admin'; config.staticAuth[0].password = 'contrail123'; config.staticAuth[0].roles = ['cloudAdmin']; INFO: config /etc/contrail/contrail-webui-userauth.js /* * Copyright (c) 2014 Juniper Networks, Inc. All rights reserved. */ var auth = {}; auth.admin_user = 'admin'; auth.admin_password = 'contrail123'; auth.admin_token = ''; auth.admin_tenant_name = 'admin'; auth.project_domain_name = 'Default'; auth.user_domain_name = 'Default'; module.exports = auth; 10/21/2024 01:54:24 AM - debug: Built Package List as:[{"pkgName":null,"parseURL.xml":["/src/serverroot/common/url.routes.js","/src/serverroot/utils/url.routes.js","/src/serverroot/web/routes/url.routes.js","/webroot/reports/qe/api/url.routes.js","/webroot/reports/udd/api/url.routes.js"],"jobProcess.xml":["/src/serverroot/jobs/core/jobsCb.api.js"],"featureList.xml":["/src/serverroot/web/core/feature.list.js"]},{"pkgName":"webController","parseURL.xml":["/webroot/common/api/url.routes.js","/webroot/config/common/api/url.routes.js","/webroot/config/dns/records/api/url.routes.js","/webroot/config/dns/servers/api/url.routes.js","/webroot/config/firewall/common/fwpolicy/api/url.routes.js","/webroot/config/infra/bgp/api/url.routes.js","/webroot/config/infra/linklocalservices/api/url.routes.js","/webroot/config/infra/quotas/api/url.routes.js","/webroot/config/infra/serviceappliance/api/url.routes.js","/webroot/config/infra/serviceapplianceset/api/url.routes.js","/webroot/config/infra/vrouters/api/url.routes.js","/webroot/config/networking/fip/api/url.routes.js","/webroot/config/networking/ipam/api/url.routes.js","/webroot/config/networking/loadbalancer/api/url.routes.js","/webroot/config/networking/logicalrouter/api/url.routes.js","/webroot/config/networking/networks/api/url.routes.js","/webroot/config/networking/policy/api/url.routes.js","/webroot/config/networking/port/api/url.routes.js","/webroot/config/networking/routetable/api/url.routes.js","/webroot/config/networking/routingpolicy/api/url.routes.js","/webroot/config/networking/securitygroup/api/url.routes.js","/webroot/config/physicaldevices/interfaces/api/url.routes.js","/webroot/config/physicaldevices/physicalrouters/api/url.routes.js","/webroot/config/port/api/url.routes.js","/webroot/config/services/bgpasaservice/api/url.routes.js","/webroot/config/services/instances/api/url.routes.js","/webroot/config/services/templates/api/url.routes.js","/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js","/webroot/monitor/infrastructure/confignode/api/url.routes.js","/webroot/monitor/infrastructure/controlnode/api/url.routes.js","/webroot/monitor/infrastructure/databasenode/api/url.routes.js","/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js","/webroot/monitor/infrastructure/underlay/api/url.routes.js","/webroot/monitor/infrastructure/vrouter/api/url.routes.js","/webroot/monitor/networking/api/url.routes.js","/webroot/setting/configdb/ui/api/url.routes.js"],"featureList.xml":["/webroot/common/api/feature.list.js"],"jobProcess.xml":["/webroot/monitor/infrastructure/controlnode/jobs/jobsCb.api.js","/webroot/monitor/infrastructure/vrouter/jobs/jobsCb.api.js","/webroot/monitor/networking/jobs/jobsCb.api.js"]}] 10/21/2024 01:54:24 AM - debug: Redis: connect 10/21/2024 01:54:24 AM - debug: Redis: connect 10/21/2024 01:54:24 AM - debug: Redis: connect 10/21/2024 01:54:24 AM - debug: Redis: connect 10/21/2024 01:54:24 AM - debug: Redis: connect 10/21/2024 01:54:24 AM - debug: Redis: ready 10/21/2024 01:54:24 AM - debug: Redis: ready 10/21/2024 01:54:24 AM - debug: Redis: ready 10/21/2024 01:54:24 AM - debug: Redis: ready 10/21/2024 01:54:24 AM - debug: Redis: ready Done, creating file: /usr/src/contrail/contrail-web-core/webroot/common/api/regions.js 10/21/2024 01:54:24 AM - debug: Web Server bound to port 3000 to Job Server 10/21/2024 01:54:24 AM - debug: Redis: ready 10/21/2024 01:54:24 AM - debug: Redis DB 2 SELECT suuccess: 10/21/2024 01:54:25 AM - debug: Built Package List as:[{"pkgName":null,"parseURL.xml":["/src/serverroot/common/url.routes.js","/src/serverroot/utils/url.routes.js","/src/serverroot/web/routes/url.routes.js","/webroot/reports/qe/api/url.routes.js","/webroot/reports/udd/api/url.routes.js"],"jobProcess.xml":["/src/serverroot/jobs/core/jobsCb.api.js"],"featureList.xml":["/src/serverroot/web/core/feature.list.js"]},{"pkgName":"webController","parseURL.xml":["/webroot/common/api/url.routes.js","/webroot/config/common/api/url.routes.js","/webroot/config/dns/records/api/url.routes.js","/webroot/config/dns/servers/api/url.routes.js","/webroot/config/firewall/common/fwpolicy/api/url.routes.js","/webroot/config/infra/bgp/api/url.routes.js","/webroot/config/infra/linklocalservices/api/url.routes.js","/webroot/config/infra/quotas/api/url.routes.js","/webroot/config/infra/serviceappliance/api/url.routes.js","/webroot/config/infra/serviceapplianceset/api/url.routes.js","/webroot/config/infra/vrouters/api/url.routes.js","/webroot/config/networking/fip/api/url.routes.js","/webroot/config/networking/ipam/api/url.routes.js","/webroot/config/networking/loadbalancer/api/url.routes.js","/webroot/config/networking/logicalrouter/api/url.routes.js","/webroot/config/networking/networks/api/url.routes.js","/webroot/config/networking/policy/api/url.routes.js","/webroot/config/networking/port/api/url.routes.js","/webroot/config/networking/routetable/api/url.routes.js","/webroot/config/networking/routingpolicy/api/url.routes.js","/webroot/config/networking/securitygroup/api/url.routes.js","/webroot/config/physicaldevices/interfaces/api/url.routes.js","/webroot/config/physicaldevices/physicalrouters/api/url.routes.js","/webroot/config/port/api/url.routes.js","/webroot/config/services/bgpasaservice/api/url.routes.js","/webroot/config/services/instances/api/url.routes.js","/webroot/config/services/templates/api/url.routes.js","/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js","/webroot/monitor/infrastructure/confignode/api/url.routes.js","/webroot/monitor/infrastructure/controlnode/api/url.routes.js","/webroot/monitor/infrastructure/databasenode/api/url.routes.js","/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js","/webroot/monitor/infrastructure/underlay/api/url.routes.js","/webroot/monitor/infrastructure/vrouter/api/url.routes.js","/webroot/monitor/networking/api/url.routes.js","/webroot/setting/configdb/ui/api/url.routes.js"],"featureList.xml":["/webroot/common/api/feature.list.js"],"jobProcess.xml":["/webroot/monitor/infrastructure/controlnode/jobs/jobsCb.api.js","/webroot/monitor/infrastructure/vrouter/jobs/jobsCb.api.js","/webroot/monitor/networking/jobs/jobsCb.api.js"]}] 10/21/2024 01:54:25 AM - debug: Redis: connect 10/21/2024 01:54:25 AM - debug: Redis: connect 10/21/2024 01:54:25 AM - debug: Redis: connect 10/21/2024 01:54:25 AM - debug: Redis: connect 10/21/2024 01:54:25 AM - debug: Redis: connect 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis DB 3 SELECT suuccess: 10/21/2024 01:54:25 AM - debug: Redis: ready 10/21/2024 01:54:25 AM - debug: Redis DB 2 SELECT suuccess: 10/21/2024 01:54:25 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-core/webroot 10/21/2024 01:54:25 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-controller/webroot 10/21/2024 01:54:25 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/common/url.routes.js 10/21/2024 01:54:25 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/utils/url.routes.js 10/21/2024 01:54:25 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/web/routes/url.routes.js 10/21/2024 01:54:25 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/qe/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/udd/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/common/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/common/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/records/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/servers/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/firewall/common/fwpolicy/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/bgp/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/linklocalservices/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/quotas/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceappliance/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceapplianceset/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/vrouters/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/fip/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/ipam/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/loadbalancer/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/logicalrouter/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/networks/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/policy/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/port/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routetable/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routingpolicy/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/securitygroup/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/interfaces/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/physicalrouters/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/port/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/bgpasaservice/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/instances/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/templates/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/confignode/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/controlnode/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/databasenode/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/underlay/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/vrouter/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/networking/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/setting/configdb/ui/api/url.routes.js 10/21/2024 01:54:26 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-core/src/serverroot/web/core/feature.list.js 10/21/2024 01:54:26 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-controller/webroot/common/api/feature.list.js 10/21/2024 01:54:26 AM - debug: **** Contrail-WebUI Server **** 10/21/2024 01:54:26 AM - debug: Redis: connect 10/21/2024 01:54:26 AM - debug: Redis: connect 10/21/2024 01:54:26 AM - debug: Redis: connect Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:26 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:26 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] { [Error: All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.] innerErrors: { '10.0.0.36:9042': { [Error: connect ECONNREFUSED] code: 'ECONNREFUSED', errno: 'ECONNREFUSED', syscall: 'connect' } }, info: 'Represents an error when a query cannot be performed because no host is available or could be reached by the driver.', message: 'All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.' } 10/21/2024 01:54:26 AM - error: Configured/Default MOTD file read error code 34 10/21/2024 01:54:26 AM - debug: Redis: ready 10/21/2024 01:54:26 AM - debug: Redis: ready 10/21/2024 01:54:26 AM - debug: Redis: ready Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:26 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:26 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:54:26 AM - debug: Contrail Service Response Updated by process:267 {"dns-server":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8092","status":"up"}]} 10/21/2024 01:54:26 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:26 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:26 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:26 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:27 AM - error: URL [http://10.0.0.36:8082/global-system-configs] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:54:27 AM - debug: Contrail Service Response Updated by process:267 {"ApiServer":[]} Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:27 AM - error: URL [http://10.0.0.36:8082/global-system-configs] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:345 data[i].status = "down"; ^ TypeError: Cannot set property 'status' of undefined at updateContrailServiceData (/usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:345:28) at /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:323:9 at /usr/lib64/node_modules/async/lib/async.js:254:17 at done (/usr/lib64/node_modules/async/lib/async.js:135:19) at /usr/lib64/node_modules/async/lib/async.js:32:16 at /usr/lib64/node_modules/async/lib/async.js:251:21 at /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:264:9 at errorback (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:235:5) at APIServer.retryMakeCall (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:225:9) at Request. (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:359:18) 10/21/2024 01:54:27 AM - debug: The worker #1 has disconnected. 10/21/2024 01:54:27 AM - error: The worker #1 has exited with exit code 8 10/21/2024 01:54:28 AM - debug: Built Package List as:[{"pkgName":null,"parseURL.xml":["/src/serverroot/common/url.routes.js","/src/serverroot/utils/url.routes.js","/src/serverroot/web/routes/url.routes.js","/webroot/reports/qe/api/url.routes.js","/webroot/reports/udd/api/url.routes.js"],"jobProcess.xml":["/src/serverroot/jobs/core/jobsCb.api.js"],"featureList.xml":["/src/serverroot/web/core/feature.list.js"]},{"pkgName":"webController","parseURL.xml":["/webroot/common/api/url.routes.js","/webroot/config/common/api/url.routes.js","/webroot/config/dns/records/api/url.routes.js","/webroot/config/dns/servers/api/url.routes.js","/webroot/config/firewall/common/fwpolicy/api/url.routes.js","/webroot/config/infra/bgp/api/url.routes.js","/webroot/config/infra/linklocalservices/api/url.routes.js","/webroot/config/infra/quotas/api/url.routes.js","/webroot/config/infra/serviceappliance/api/url.routes.js","/webroot/config/infra/serviceapplianceset/api/url.routes.js","/webroot/config/infra/vrouters/api/url.routes.js","/webroot/config/networking/fip/api/url.routes.js","/webroot/config/networking/ipam/api/url.routes.js","/webroot/config/networking/loadbalancer/api/url.routes.js","/webroot/config/networking/logicalrouter/api/url.routes.js","/webroot/config/networking/networks/api/url.routes.js","/webroot/config/networking/policy/api/url.routes.js","/webroot/config/networking/port/api/url.routes.js","/webroot/config/networking/routetable/api/url.routes.js","/webroot/config/networking/routingpolicy/api/url.routes.js","/webroot/config/networking/securitygroup/api/url.routes.js","/webroot/config/physicaldevices/interfaces/api/url.routes.js","/webroot/config/physicaldevices/physicalrouters/api/url.routes.js","/webroot/config/port/api/url.routes.js","/webroot/config/services/bgpasaservice/api/url.routes.js","/webroot/config/services/instances/api/url.routes.js","/webroot/config/services/templates/api/url.routes.js","/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js","/webroot/monitor/infrastructure/confignode/api/url.routes.js","/webroot/monitor/infrastructure/controlnode/api/url.routes.js","/webroot/monitor/infrastructure/databasenode/api/url.routes.js","/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js","/webroot/monitor/infrastructure/underlay/api/url.routes.js","/webroot/monitor/infrastructure/vrouter/api/url.routes.js","/webroot/monitor/networking/api/url.routes.js","/webroot/setting/configdb/ui/api/url.routes.js"],"featureList.xml":["/webroot/common/api/feature.list.js"],"jobProcess.xml":["/webroot/monitor/infrastructure/controlnode/jobs/jobsCb.api.js","/webroot/monitor/infrastructure/vrouter/jobs/jobsCb.api.js","/webroot/monitor/networking/jobs/jobsCb.api.js"]}] 10/21/2024 01:54:28 AM - debug: Redis: connect 10/21/2024 01:54:28 AM - debug: Redis: connect 10/21/2024 01:54:28 AM - debug: Redis: connect 10/21/2024 01:54:28 AM - debug: Redis: connect 10/21/2024 01:54:28 AM - debug: Redis: connect 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis DB 3 SELECT suuccess: 10/21/2024 01:54:28 AM - debug: Redis: ready 10/21/2024 01:54:28 AM - debug: Redis DB 2 SELECT suuccess: 10/21/2024 01:54:28 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-core/webroot 10/21/2024 01:54:28 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-controller/webroot 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/common/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/utils/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/web/routes/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/qe/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/udd/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/common/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/common/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/records/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/servers/api/url.routes.js 10/21/2024 01:54:28 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/firewall/common/fwpolicy/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/bgp/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/linklocalservices/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/quotas/api/url.routes.js 10/21/2024 01:54:29 AM - error: Worker taking too long to start. 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceappliance/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceapplianceset/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/vrouters/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/fip/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/ipam/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/loadbalancer/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/logicalrouter/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/networks/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/policy/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/port/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routetable/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routingpolicy/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/securitygroup/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/interfaces/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/physicalrouters/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/port/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/bgpasaservice/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/instances/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/templates/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/confignode/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/controlnode/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/databasenode/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/underlay/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/vrouter/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/networking/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/setting/configdb/ui/api/url.routes.js 10/21/2024 01:54:29 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-core/src/serverroot/web/core/feature.list.js 10/21/2024 01:54:29 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-controller/webroot/common/api/feature.list.js 10/21/2024 01:54:29 AM - debug: **** Contrail-WebUI Server **** 10/21/2024 01:54:29 AM - debug: Redis: connect 10/21/2024 01:54:29 AM - debug: Redis: connect 10/21/2024 01:54:29 AM - debug: Redis: connect Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] { [Error: All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.] innerErrors: { '10.0.0.36:9042': { [Error: connect ECONNREFUSED] code: 'ECONNREFUSED', errno: 'ECONNREFUSED', syscall: 'connect' } }, info: 'Represents an error when a query cannot be performed because no host is available or could be reached by the driver.', message: 'All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.' } 10/21/2024 01:54:29 AM - error: Configured/Default MOTD file read error code 34 10/21/2024 01:54:29 AM - debug: Redis: ready 10/21/2024 01:54:29 AM - debug: Redis: ready 10/21/2024 01:54:29 AM - debug: Redis: ready Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:54:29 AM - debug: Contrail Service Response Updated by process:273 {"dns-server":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8092","status":"up"}]} 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:29 AM - debug: Contrail Service Response Updated by process:273 {"OpServer":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8081","status":"up"}]} Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8082/global-system-configs] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:54:29 AM - debug: Contrail Service Response Updated by process:273 {"ApiServer":[]} 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:29 AM - error: URL [http://10.0.0.36:8082/global-system-configs] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:345 data[i].status = "down"; ^ TypeError: Cannot set property 'status' of undefined at updateContrailServiceData (/usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:345:28) at /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:323:9 at /usr/lib64/node_modules/async/lib/async.js:254:17 at done (/usr/lib64/node_modules/async/lib/async.js:135:19) at /usr/lib64/node_modules/async/lib/async.js:32:16 at /usr/lib64/node_modules/async/lib/async.js:251:21 at /usr/src/contrail/contrail-web-core/src/serverroot/common/contrailservice.api.js:264:9 at errorback (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:235:5) at APIServer.retryMakeCall (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:225:9) at Request. (/usr/src/contrail/contrail-web-core/src/serverroot/common/rest.api.js:359:18) 10/21/2024 01:54:29 AM - debug: The worker #2 has disconnected. 10/21/2024 01:54:29 AM - error: The worker #2 has exited with exit code 8 10/21/2024 01:54:31 AM - debug: Built Package List as:[{"pkgName":null,"parseURL.xml":["/src/serverroot/common/url.routes.js","/src/serverroot/utils/url.routes.js","/src/serverroot/web/routes/url.routes.js","/webroot/reports/qe/api/url.routes.js","/webroot/reports/udd/api/url.routes.js"],"jobProcess.xml":["/src/serverroot/jobs/core/jobsCb.api.js"],"featureList.xml":["/src/serverroot/web/core/feature.list.js"]},{"pkgName":"webController","parseURL.xml":["/webroot/common/api/url.routes.js","/webroot/config/common/api/url.routes.js","/webroot/config/dns/records/api/url.routes.js","/webroot/config/dns/servers/api/url.routes.js","/webroot/config/firewall/common/fwpolicy/api/url.routes.js","/webroot/config/infra/bgp/api/url.routes.js","/webroot/config/infra/linklocalservices/api/url.routes.js","/webroot/config/infra/quotas/api/url.routes.js","/webroot/config/infra/serviceappliance/api/url.routes.js","/webroot/config/infra/serviceapplianceset/api/url.routes.js","/webroot/config/infra/vrouters/api/url.routes.js","/webroot/config/networking/fip/api/url.routes.js","/webroot/config/networking/ipam/api/url.routes.js","/webroot/config/networking/loadbalancer/api/url.routes.js","/webroot/config/networking/logicalrouter/api/url.routes.js","/webroot/config/networking/networks/api/url.routes.js","/webroot/config/networking/policy/api/url.routes.js","/webroot/config/networking/port/api/url.routes.js","/webroot/config/networking/routetable/api/url.routes.js","/webroot/config/networking/routingpolicy/api/url.routes.js","/webroot/config/networking/securitygroup/api/url.routes.js","/webroot/config/physicaldevices/interfaces/api/url.routes.js","/webroot/config/physicaldevices/physicalrouters/api/url.routes.js","/webroot/config/port/api/url.routes.js","/webroot/config/services/bgpasaservice/api/url.routes.js","/webroot/config/services/instances/api/url.routes.js","/webroot/config/services/templates/api/url.routes.js","/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js","/webroot/monitor/infrastructure/confignode/api/url.routes.js","/webroot/monitor/infrastructure/controlnode/api/url.routes.js","/webroot/monitor/infrastructure/databasenode/api/url.routes.js","/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js","/webroot/monitor/infrastructure/underlay/api/url.routes.js","/webroot/monitor/infrastructure/vrouter/api/url.routes.js","/webroot/monitor/networking/api/url.routes.js","/webroot/setting/configdb/ui/api/url.routes.js"],"featureList.xml":["/webroot/common/api/feature.list.js"],"jobProcess.xml":["/webroot/monitor/infrastructure/controlnode/jobs/jobsCb.api.js","/webroot/monitor/infrastructure/vrouter/jobs/jobsCb.api.js","/webroot/monitor/networking/jobs/jobsCb.api.js"]}] 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis DB 3 SELECT suuccess: 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis DB 2 SELECT suuccess: 10/21/2024 01:54:31 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-core/webroot 10/21/2024 01:54:31 AM - debug: Registering Static Directory: /usr/src/contrail/contrail-web-controller/webroot 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/common/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/utils/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/src/serverroot/web/routes/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/qe/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-core/webroot/reports/udd/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/common/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/common/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/records/api/url.routes.js 10/21/2024 01:54:31 AM - error: Worker taking too long to start. 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/dns/servers/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/firewall/common/fwpolicy/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/bgp/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/linklocalservices/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/quotas/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceappliance/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/serviceapplianceset/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/infra/vrouters/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/fip/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/ipam/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/loadbalancer/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/logicalrouter/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/networks/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/policy/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/port/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routetable/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/routingpolicy/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/networking/securitygroup/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/interfaces/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/physicaldevices/physicalrouters/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/port/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/bgpasaservice/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/instances/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/config/services/templates/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/analyticsnode/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/confignode/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/controlnode/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/databasenode/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/globalcontroller/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/underlay/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/infrastructure/vrouter/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/monitor/networking/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registered URLs to APP :/usr/src/contrail/contrail-web-controller/webroot/setting/configdb/ui/api/url.routes.js 10/21/2024 01:54:31 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-core/src/serverroot/web/core/feature.list.js 10/21/2024 01:54:31 AM - debug: Registering feature Lists: /usr/src/contrail/contrail-web-controller/webroot/common/api/feature.list.js 10/21/2024 01:54:31 AM - debug: **** Contrail-WebUI Server **** 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect 10/21/2024 01:54:31 AM - debug: Redis: connect Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] { [Error: All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.] innerErrors: { '10.0.0.36:9042': { [Error: connect ECONNREFUSED] code: 'ECONNREFUSED', errno: 'ECONNREFUSED', syscall: 'connect' } }, info: 'Represents an error when a query cannot be performed because no host is available or could be reached by the driver.', message: 'All host(s) tried for query failed. First host tried, 10.0.0.36:9042: Error: connect ECONNREFUSED. See innerErrors.' } 10/21/2024 01:54:31 AM - error: Configured/Default MOTD file read error code 34 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready 10/21/2024 01:54:31 AM - debug: Redis: ready Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:54:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:54:31 AM - debug: Contrail Service Response Updated by process:279 {"dns-server":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8092","status":"up"}]} 10/21/2024 01:54:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:54:31 AM - debug: Contrail Service Response Updated by process:279 {"OpServer":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8081","status":"up"}]} 10/21/2024 01:54:31 AM - debug: Contrail Service Response Updated by process:279 {"ApiServer":[{"@publisher-id":"10.0.0.36","ip-address":"10.0.0.36","port":"8082","status":"up"}]} Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:59:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:59:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:59:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 01:59:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 01:59:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:59:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:59:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 01:59:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 02:04:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 02:04:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 02:04:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] Error: connect ECONNREFUSED at errnoException (net.js:905:11) at Object.afterConnect [as oncomplete] (net.js:896:19) 10/21/2024 02:04:31 AM - error: URL [http://127.0.0.1:5000/v3/auth/tokens] returned error [{"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect"}] 10/21/2024 02:04:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 02:04:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 02:04:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"] 10/21/2024 02:04:31 AM - error: URL [http://10.0.0.36:8092/v3/auth/tokens] returned error ["\n404 Not Found\n\r\n\u0000"]